Max level shown:
1 [ Aug 22 17:49:12 Enabled. ]
2 [ Aug 22 17:49:12 Rereading configuration. ]
3 [ Aug 22 17:49:15 Rereading configuration. ]
4 [ Aug 22 17:49:32 Executing start method ("ctrun -l child -o noorphan,regent /opt/oxide/omicron-nexus/bin/nexus /var/svc/manifest/site/nexus/config.toml &"). ]
5 [ Aug 22 17:49:32 Method "start" exited with status 0. ]
6 note: configured to log to "/dev/stdout"
72024-08-22T17:49:32.484ZDEBGnexus: registered DTrace probes
82024-08-22T17:49:32.485ZINFOnexus: setting up nexus server file = nexus/src/lib.rs:86
92024-08-22T17:49:32.493ZINFOnexus (ServerContext): registering Oso class class = Action file = nexus/auth/src/authz/oso_generic.rs:68
102024-08-22T17:49:32.493ZINFOnexus (ServerContext): registering Oso class class = AnyActor file = nexus/auth/src/authz/oso_generic.rs:68
112024-08-22T17:49:32.493ZINFOnexus (ServerContext): registering Oso class class = AuthenticatedActor file = nexus/auth/src/authz/oso_generic.rs:68
122024-08-22T17:49:32.493ZINFOnexus (ServerContext): registering Oso class class = BlueprintConfig file = nexus/auth/src/authz/oso_generic.rs:68
132024-08-22T17:49:32.493ZINFOnexus (ServerContext): registering Oso class class = Database file = nexus/auth/src/authz/oso_generic.rs:68
142024-08-22T17:49:32.493ZINFOnexus (ServerContext): registering Oso class class = DnsConfig file = nexus/auth/src/authz/oso_generic.rs:68
152024-08-22T17:49:32.493ZINFOnexus (ServerContext): registering Oso class class = Fleet file = nexus/auth/src/authz/oso_generic.rs:68
162024-08-22T17:49:32.493ZINFOnexus (ServerContext): registering Oso class class = Inventory file = nexus/auth/src/authz/oso_generic.rs:68
172024-08-22T17:49:32.493ZINFOnexus (ServerContext): registering Oso class class = IpPoolList file = nexus/auth/src/authz/oso_generic.rs:68
182024-08-22T17:49:32.493ZINFOnexus (ServerContext): registering Oso class class = ConsoleSessionList file = nexus/auth/src/authz/oso_generic.rs:68
192024-08-22T17:49:32.493ZINFOnexus (ServerContext): registering Oso class class = DeviceAuthRequestList file = nexus/auth/src/authz/oso_generic.rs:68
202024-08-22T17:49:32.493ZINFOnexus (ServerContext): registering Oso class class = SiloCertificateList file = nexus/auth/src/authz/oso_generic.rs:68
212024-08-22T17:49:32.493ZINFOnexus (ServerContext): registering Oso class class = SiloIdentityProviderList file = nexus/auth/src/authz/oso_generic.rs:68
222024-08-22T17:49:32.493ZINFOnexus (ServerContext): registering Oso class class = SiloUserList file = nexus/auth/src/authz/oso_generic.rs:68
232024-08-22T17:49:32.493ZINFOnexus (ServerContext): registering Oso class class = Project file = nexus/auth/src/authz/oso_generic.rs:68
242024-08-22T17:49:32.493ZINFOnexus (ServerContext): registering Oso class class = Disk file = nexus/auth/src/authz/oso_generic.rs:68
252024-08-22T17:49:32.493ZINFOnexus (ServerContext): registering Oso class class = Snapshot file = nexus/auth/src/authz/oso_generic.rs:68
262024-08-22T17:49:32.493ZINFOnexus (ServerContext): registering Oso class class = ProjectImage file = nexus/auth/src/authz/oso_generic.rs:68
272024-08-22T17:49:32.493ZINFOnexus (ServerContext): registering Oso class class = Instance file = nexus/auth/src/authz/oso_generic.rs:68
282024-08-22T17:49:32.493ZINFOnexus (ServerContext): registering Oso class class = IpPool file = nexus/auth/src/authz/oso_generic.rs:68
292024-08-22T17:49:32.493ZINFOnexus (ServerContext): registering Oso class class = InstanceNetworkInterface file = nexus/auth/src/authz/oso_generic.rs:68
302024-08-22T17:49:32.493ZINFOnexus (ServerContext): registering Oso class class = Vpc file = nexus/auth/src/authz/oso_generic.rs:68
312024-08-22T17:49:32.493ZINFOnexus (ServerContext): registering Oso class class = VpcRouter file = nexus/auth/src/authz/oso_generic.rs:68
322024-08-22T17:49:32.493ZINFOnexus (ServerContext): registering Oso class class = RouterRoute file = nexus/auth/src/authz/oso_generic.rs:68
332024-08-22T17:49:32.493ZINFOnexus (ServerContext): registering Oso class class = VpcSubnet file = nexus/auth/src/authz/oso_generic.rs:68
342024-08-22T17:49:32.493ZINFOnexus (ServerContext): registering Oso class class = FloatingIp file = nexus/auth/src/authz/oso_generic.rs:68
352024-08-22T17:49:32.493ZINFOnexus (ServerContext): registering Oso class class = Image file = nexus/auth/src/authz/oso_generic.rs:68
362024-08-22T17:49:32.493ZINFOnexus (ServerContext): registering Oso class class = SiloImage file = nexus/auth/src/authz/oso_generic.rs:68
372024-08-22T17:49:32.493ZINFOnexus (ServerContext): registering Oso class class = AddressLot file = nexus/auth/src/authz/oso_generic.rs:68
382024-08-22T17:49:32.493ZINFOnexus (ServerContext): registering Oso class class = Blueprint file = nexus/auth/src/authz/oso_generic.rs:68
392024-08-22T17:49:32.493ZINFOnexus (ServerContext): registering Oso class class = LoopbackAddress file = nexus/auth/src/authz/oso_generic.rs:68
402024-08-22T17:49:32.493ZINFOnexus (ServerContext): registering Oso class class = Certificate file = nexus/auth/src/authz/oso_generic.rs:68
412024-08-22T17:49:32.493ZINFOnexus (ServerContext): registering Oso class class = ConsoleSession file = nexus/auth/src/authz/oso_generic.rs:68
422024-08-22T17:49:32.493ZINFOnexus (ServerContext): registering Oso class class = DeviceAuthRequest file = nexus/auth/src/authz/oso_generic.rs:68
432024-08-22T17:49:32.493ZINFOnexus (ServerContext): registering Oso class class = DeviceAccessToken file = nexus/auth/src/authz/oso_generic.rs:68
442024-08-22T17:49:32.493ZINFOnexus (ServerContext): registering Oso class class = PhysicalDisk file = nexus/auth/src/authz/oso_generic.rs:68
452024-08-22T17:49:32.493ZINFOnexus (ServerContext): registering Oso class class = Rack file = nexus/auth/src/authz/oso_generic.rs:68
462024-08-22T17:49:32.493ZINFOnexus (ServerContext): registering Oso class class = RoleBuiltin file = nexus/auth/src/authz/oso_generic.rs:68
472024-08-22T17:49:32.493ZINFOnexus (ServerContext): registering Oso class class = SshKey file = nexus/auth/src/authz/oso_generic.rs:68
482024-08-22T17:49:32.493ZINFOnexus (ServerContext): registering Oso class class = Silo file = nexus/auth/src/authz/oso_generic.rs:68
492024-08-22T17:49:32.493ZINFOnexus (ServerContext): registering Oso class class = SiloUser file = nexus/auth/src/authz/oso_generic.rs:68
502024-08-22T17:49:32.493ZINFOnexus (ServerContext): registering Oso class class = SiloGroup file = nexus/auth/src/authz/oso_generic.rs:68
512024-08-22T17:49:32.493ZINFOnexus (ServerContext): registering Oso class class = IdentityProvider file = nexus/auth/src/authz/oso_generic.rs:68
522024-08-22T17:49:32.493ZINFOnexus (ServerContext): registering Oso class class = SamlIdentityProvider file = nexus/auth/src/authz/oso_generic.rs:68
532024-08-22T17:49:32.493ZINFOnexus (ServerContext): registering Oso class class = Sled file = nexus/auth/src/authz/oso_generic.rs:68
542024-08-22T17:49:32.493ZINFOnexus (ServerContext): registering Oso class class = TufRepo file = nexus/auth/src/authz/oso_generic.rs:68
552024-08-22T17:49:32.493ZINFOnexus (ServerContext): registering Oso class class = TufArtifact file = nexus/auth/src/authz/oso_generic.rs:68
562024-08-22T17:49:32.493ZINFOnexus (ServerContext): registering Oso class class = Zpool file = nexus/auth/src/authz/oso_generic.rs:68
572024-08-22T17:49:32.493ZINFOnexus (ServerContext): registering Oso class class = Service file = nexus/auth/src/authz/oso_generic.rs:68
582024-08-22T17:49:32.493ZINFOnexus (ServerContext): registering Oso class class = UserBuiltin file = nexus/auth/src/authz/oso_generic.rs:68
592024-08-22T17:49:32.493ZINFOnexus (ServerContext): full Oso configuration config = #\n# Oso configuration for Omicron\n# This file is augmented by generated snippets.\n#\n\n#\n# ACTOR TYPES AND BASIC RULES\n#\n\n# `AnyActor` includes both authenticated and unauthenticated users.\nactor AnyActor {}\n\n# An `AuthenticatedActor` has an identity in the system. All of our operations\n# today require that an actor be authenticated.\nactor AuthenticatedActor {}\n\n# For any resource, `actor` can perform action `action` on it if they're\n# authenticated and their role(s) give them the corresponding permission on that\n# resource.\nallow(actor: AnyActor, action: Action, resource) if\n actor.authenticated and\n has_permission(actor.authn_actor.unwrap(), action.to_perm(), resource);\n\n# Define role relationships\nhas_role(actor: AuthenticatedActor, role: String, resource: Resource)\n\tif resource.has_role(actor, role);\n\n#\n# ROLES AND PERMISSIONS IN THE FLEET/SILO/PROJECT HIERARCHY\n#\n# We define the following permissions for most resources in the system:\n#\n# - "create_child": required to create child resources (of any type)\n#\n# - "list_children": required to list child resources (of all types) of a\n# resource\n#\n# - "modify": required to modify or delete a resource\n#\n# - "read": required to read a resource\n#\n# We define the following predefined roles for only a few high-level resources:\n# the Fleet (see below), Silo, Organization, and Project. The specific roles\n# are oriented around intended use-cases:\n#\n# - "admin": has all permissions on the resource\n#\n# - "collaborator": has "read", "list_children", and "create_child", plus\n# the "admin" role for child resources. The idea is that if you're an\n# Organization Collaborator, you have full control over the Projects within\n# the Organization, but you cannot modify or delete the Organization itself.\n#\n# - "viewer": has "read" and "list_children" on a resource\n#\n# Below the Project level, permissions are granted via roles at the Project\n# level. For example, for someone to be able to create, modify, or delete any\n# Instances, they must be granted project.collaborator, which means they can\n# create, modify, or delete _all_ resources in the Project.\n#\n# The complete set of predefined roles:\n#\n# - fleet.admin (superuser for the whole system)\n# - fleet.collaborator (can manage Silos)\n# - fleet.viewer (can read most non-siloed resources in the system)\n# - silo.admin (superuser for the silo)\n# - silo.collaborator (can create and own Organizations)\n# - silo.viewer (can read most resources within the Silo)\n# - organization.admin (complete control over an organization)\n# - organization.collaborator (can manage Projects)\n# - organization.viewer (can read most resources within the Organization)\n# - project.admin (complete control over a Project)\n# - project.collaborator (can manage all resources within the Project)\n# - project.viewer (can read most resources within the Project)\n#\n# Outside the Silo/Organization/Project hierarchy, we (currently) treat most\n# resources as nested under Fleet or else a synthetic resource (see below). We\n# do not yet support role assignments on anything other than Fleet, Silo,\n# Organization, or Project.\n#\n\n# "Fleet" is a global singleton representing the whole system. The name comes\n# from the idea described in RFD 24, but it's not quite right. This probably\n# should be more like "Region" or "AvailabilityZone". The precise boundaries\n# have not yet been figured out.\nresource Fleet {\n\tpermissions = [\n\t "list_children",\n\t "modify",\n\t "read",\n\t "create_child",\n\t];\n\n\troles = [\n\t # Roles that can be attached by users\n\t "admin",\n\t "collaborator",\n\t "viewer",\n\n\t # Internal-only roles\n\t "external-authenticator"\n\t];\n\n\t# Roles implied by other roles on this resource\n\t"viewer" if "collaborator";\n\t"collaborator" if "admin";\n\n\t# Permissions granted directly by roles on this resource\n\t"list_children" if "viewer";\n\t"read" if "viewer";\n\t"create_child" if "collaborator";\n\t"modify" if "admin";\n}\n\n# For fleets specifically, roles can be conferred by roles on the user's Silo.\nhas_role(actor: AuthenticatedActor, role: String, _: Fleet) if\n\tsilo_role in actor.confers_fleet_role(role) and\n\thas_role(actor, silo_role, actor.silo.unwrap());\n\nresource Silo {\n\tpermissions = [\n\t "list_children",\n\t "modify",\n\t "read",\n\t "create_child",\n\t];\n\troles = [ "admin", "collaborator", "viewer" ];\n\n\t# Roles implied by other roles on this resource\n\t"viewer" if "collaborator";\n\t"collaborator" if "admin";\n\n\t# Permissions granted directly by roles on this resource\n\t"list_children" if "viewer";\n\t"read" if "viewer";\n\n\t"create_child" if "collaborator";\n\t"modify" if "admin";\n\n\t# Permissions implied by roles on this resource's parent (Fleet). Fleet\n\t# privileges allow a user to see and potentially administer the Silo,\n\t# but they do not give anyone permission to look at anything inside the\n\t# Silo. To achieve this, we use permission rules here. (If we granted\n\t# Fleet administrators _roles_ on the Silo, then those would cascade\n\t# into the Silo as well.)\n\trelations = { parent_fleet: Fleet };\n\t"read" if "viewer" on "parent_fleet";\n\t"modify" if "collaborator" on "parent_fleet";\n\n\t# external authenticator has to create silo users\n\t"list_children" if "external-authenticator" on "parent_fleet";\n\t"create_child" if "external-authenticator" on "parent_fleet";\n}\n\nhas_relation(fleet: Fleet, "parent_fleet", silo: Silo)\n\tif silo.fleet = fleet;\n\n# As a special case, all authenticated users can read their own Silo. That's\n# not quite the same as having the "viewer" role. For example, they cannot list\n# Organizations in the Silo.\n#\n# One reason this is necessary is because if an unprivileged user tries to\n# create an Organization using "POST /organizations", they should get back a 403\n# (which implies they're able to see /organizations, which is essentially seeing\n# the Silo itself) rather than a 404. This behavior isn't a hard constraint\n# (i.e., you could reasonably get a 404 for an API you're not allowed to call).\n# Nor is the implementation (i.e., we could special-case this endpoint somehow).\n# But granting this permission is the simplest way to keep this endpoint's\n# behavior consistent with the rest of the API.\n#\n# This rule is also used to determine if a user can list the identity providers\n# in the Silo (which they should be able to), since that's predicated on being\n# able to read the Silo.\n#\n# It's unclear what else would break if users couldn't see their own Silo.\nhas_permission(actor: AuthenticatedActor, "read", silo: Silo)\n\tif silo in actor.silo;\n\nresource Project {\n\tpermissions = [\n\t "list_children",\n\t "modify",\n\t "read",\n\t "create_child",\n\t];\n\troles = [ "admin", "collaborator", "viewer" ];\n\n\t# Roles implied by other roles on this resource\n\t"viewer" if "collaborator";\n\t"collaborator" if "admin";\n\n\t# Permissions granted directly by roles on this resource\n\t"list_children" if "viewer";\n\t"read" if "viewer";\n\t"create_child" if "collaborator";\n\t"modify" if "admin";\n\n\t# Roles implied by roles on this resource's parent (Silo)\n\trelations = { parent_silo: Silo };\n\t"admin" if "collaborator" on "parent_silo";\n\t"viewer" if "viewer" on "parent_silo";\n}\nhas_relation(silo: Silo, "parent_silo", project: Project)\n\tif project.silo = silo;\n\n#\n# GENERAL RESOURCES OUTSIDE THE SILO/PROJECT HIERARCHY\n#\n# Many resources use snippets of Polar generated by the `authz_resource!` Rust\n# macro. Some resources require custom Polar code. Those appear here.\n#\n\nresource Certificate {\n\tpermissions = [ "read", "modify" ];\n\trelations = { parent_silo: Silo, parent_fleet: Fleet };\n\n\t# Fleet-level and silo-level roles both grant privileges on certificates.\n\t"read" if "admin" on "parent_silo";\n\t"modify" if "admin" on "parent_silo";\n\t"read" if "admin" on "parent_fleet";\n\t"modify" if "admin" on "parent_fleet";\n}\nhas_relation(silo: Silo, "parent_silo", certificate: Certificate)\n\tif certificate.silo = silo;\nhas_relation(fleet: Fleet, "parent_fleet", certificate: Certificate)\n\tif certificate.silo.fleet = fleet;\n\nresource SiloUser {\n\tpermissions = [\n\t "list_children",\n\t "modify",\n\t "read",\n\t "create_child",\n\t];\n\n\t# Fleet and Silo administrators can manage a Silo's users. This is one\n\t# of the only areas of Silo configuration that Fleet Administrators have\n\t# permissions on.\n\trelations = { parent_silo: Silo, parent_fleet: Fleet };\n\t"list_children" if "read" on "parent_silo";\n\t"read" if "read" on "parent_silo";\n\t"modify" if "admin" on "parent_silo";\n\t"create_child" if "admin" on "parent_silo";\n\t"list_children" if "read" on "parent_fleet";\n\t"read" if "read" on "parent_fleet";\n\t"modify" if "admin" on "parent_fleet";\n\t"create_child" if "admin" on "parent_fleet";\n}\nhas_relation(silo: Silo, "parent_silo", user: SiloUser)\n\tif user.silo = silo;\nhas_relation(fleet: Fleet, "parent_fleet", user: SiloUser)\n\tif user.silo.fleet = fleet;\n\n# authenticated actors have all permissions on themselves\nhas_permission(actor: AuthenticatedActor, _perm: String, silo_user: SiloUser)\n if actor.equals_silo_user(silo_user);\n\nhas_permission(actor: AuthenticatedActor, "read", silo_user: SiloUser)\n if silo_user.silo in actor.silo;\n\nresource SiloGroup {\n\tpermissions = [\n\t "list_children",\n\t "modify",\n\t "read",\n\t "create_child",\n\t];\n\n\trelations = { parent_silo: Silo };\n\t"list_children" if "read" on "parent_silo";\n\t"read" if "read" on "parent_silo";\n\t"modify" if "admin" on "parent_silo";\n\t"create_child" if "admin" on "parent_silo";\n}\nhas_relation(silo: Silo, "parent_silo", group: SiloGroup)\n\tif group.silo = silo;\n\nresource SshKey {\n\tpermissions = [ "read", "modify" ];\n\trelations = { silo_user: SiloUser };\n\n\t"read" if "read" on "silo_user";\n\t"modify" if "modify" on "silo_user";\n}\nhas_relation(user: SiloUser, "silo_user", ssh_key: SshKey)\n\tif ssh_key.silo_user = user;\n\nresource IdentityProvider {\n\tpermissions = [\n\t "read",\n\t "modify",\n\t "create_child",\n\t "list_children",\n\t];\n\trelations = { parent_silo: Silo, parent_fleet: Fleet };\n\n\t# Silo-level roles grant privileges on identity providers.\n\t"read" if "viewer" on "parent_silo";\n\t"list_children" if "viewer" on "parent_silo";\n\t"modify" if "admin" on "parent_silo";\n\t"create_child" if "admin" on "parent_silo";\n\n\t# Fleet-level roles also grant privileges on identity providers.\n\t"read" if "viewer" on "parent_fleet";\n\t"list_children" if "viewer" on "parent_fleet";\n\t"modify" if "admin" on "parent_fleet";\n\t"create_child" if "admin" on "parent_fleet";\n}\nhas_relation(silo: Silo, "parent_silo", identity_provider: IdentityProvider)\n\tif identity_provider.silo = silo;\nhas_relation(fleet: Fleet, "parent_fleet", collection: IdentityProvider)\n\tif collection.silo.fleet = fleet;\n\nresource SamlIdentityProvider {\n\tpermissions = [\n\t "read",\n\t "modify",\n\t "create_child",\n\t "list_children",\n\t];\n\trelations = { parent_silo: Silo, parent_fleet: Fleet };\n\n\t# Silo-level roles grant privileges on identity providers.\n\t"read" if "viewer" on "parent_silo";\n\t"list_children" if "viewer" on "parent_silo";\n\t"modify" if "admin" on "parent_silo";\n\t"create_child" if "admin" on "parent_silo";\n\n\t# Fleet-level roles also grant privileges on identity providers.\n\t"read" if "viewer" on "parent_fleet";\n\t"list_children" if "viewer" on "parent_fleet";\n\t"modify" if "admin" on "parent_fleet";\n\t"create_child" if "admin" on "parent_fleet";\n}\nhas_relation(silo: Silo, "parent_silo", saml_identity_provider: SamlIdentityProvider)\n\tif saml_identity_provider.silo = silo;\nhas_relation(fleet: Fleet, "parent_fleet", collection: SamlIdentityProvider)\n\tif collection.silo.fleet = fleet;\n\n#\n# SYNTHETIC RESOURCES OUTSIDE THE SILO HIERARCHY\n#\n# The resources here do not correspond to anything that appears explicitly in\n# the API or is stored in the database. These are used either at the top level\n# of the API path (e.g., "/v1/system/ip-pools") or as an implementation detail of the system\n# (in the case of console sessions and "Database"). The policies are\n# either statically-defined in this file or driven by role assignments on the\n# Fleet. None of these resources defines their own roles.\n#\n\n# Describes the policy for reading and modifying DNS configuration\n# (both internal and external)\nresource DnsConfig {\n\tpermissions = [ "read", "modify" ];\n\trelations = { parent_fleet: Fleet };\n\t# "external-authenticator" requires these permissions because that's the\n\t# context that Nexus uses when creating and deleting Silos. These\n\t# operations necessarily need to read and modify DNS configuration.\n\t"read" if "external-authenticator" on "parent_fleet";\n\t"modify" if "external-authenticator" on "parent_fleet";\n\t# "admin" on the parent fleet also gets these permissions, primarily for\n\t# the test suite.\n\t"read" if "admin" on "parent_fleet";\n\t"modify" if "admin" on "parent_fleet";\n}\nhas_relation(fleet: Fleet, "parent_fleet", dns_config: DnsConfig)\n\tif dns_config.fleet = fleet;\n\n# Describes the policy for accessing blueprints\nresource BlueprintConfig {\n\tpermissions = [\n\t "list_children", # list blueprints\n\t "create_child", # create blueprint\n\t "read", # read the current target\n\t "modify", # change the current target\n\t];\n\n\trelations = { parent_fleet: Fleet };\n\t"create_child" if "admin" on "parent_fleet";\n\t"modify" if "admin" on "parent_fleet";\n\t"list_children" if "viewer" on "parent_fleet";\n\t"read" if "viewer" on "parent_fleet";\n}\nhas_relation(fleet: Fleet, "parent_fleet", list: BlueprintConfig)\n\tif list.fleet = fleet;\n\n# Describes the policy for reading and modifying low-level inventory\nresource Inventory {\n\tpermissions = [ "read", "modify" ];\n\trelations = { parent_fleet: Fleet };\n\t"read" if "viewer" on "parent_fleet";\n\t"modify" if "admin" on "parent_fleet";\n}\nhas_relation(fleet: Fleet, "parent_fleet", inventory: Inventory)\n\tif inventory.fleet = fleet;\n\n# Describes the policy for accessing "/v1/system/ip-pools" in the API\nresource IpPoolList {\n\tpermissions = [\n\t "list_children",\n\t "modify",\n\t "create_child",\n\t];\n\n\t# Fleet Administrators can create or modify the IP Pools list.\n\trelations = { parent_fleet: Fleet };\n\t"modify" if "admin" on "parent_fleet";\n\t"create_child" if "admin" on "parent_fleet";\n\n\t# Fleet Viewers can list IP Pools\n\t"list_children" if "viewer" on "parent_fleet";\n}\nhas_relation(fleet: Fleet, "parent_fleet", ip_pool_list: IpPoolList)\n\tif ip_pool_list.fleet = fleet;\n\n# Any authenticated user can create a child of a provided IP Pool.\n# This is necessary to use the pools when provisioning instances.\nhas_permission(actor: AuthenticatedActor, "create_child", ip_pool: IpPool)\n\tif silo in actor.silo and silo.fleet = ip_pool.fleet;\n\n# Describes the policy for creating and managing web console sessions.\nresource ConsoleSessionList {\n\tpermissions = [ "create_child" ];\n\trelations = { parent_fleet: Fleet };\n\t"create_child" if "external-authenticator" on "parent_fleet";\n}\nhas_relation(fleet: Fleet, "parent_fleet", collection: ConsoleSessionList)\n\tif collection.fleet = fleet;\n\n# Describes the policy for creating and managing device authorization requests.\nresource DeviceAuthRequestList {\n\tpermissions = [ "create_child" ];\n\trelations = { parent_fleet: Fleet };\n\t"create_child" if "external-authenticator" on "parent_fleet";\n}\nhas_relation(fleet: Fleet, "parent_fleet", collection: DeviceAuthRequestList)\n\tif collection.fleet = fleet;\n\n# Describes the policy for creating and managing Silo certificates\nresource SiloCertificateList {\n\tpermissions = [ "list_children", "create_child" ];\n\n\trelations = { parent_silo: Silo, parent_fleet: Fleet };\n\n\t# Both Fleet and Silo administrators can see and modify the Silo's\n\t# certificates.\n\t"list_children" if "admin" on "parent_silo";\n\t"list_children" if "admin" on "parent_fleet";\n\t"create_child" if "admin" on "parent_silo";\n\t"create_child" if "admin" on "parent_fleet";\n}\nhas_relation(silo: Silo, "parent_silo", collection: SiloCertificateList)\n\tif collection.silo = silo;\nhas_relation(fleet: Fleet, "parent_fleet", collection: SiloCertificateList)\n\tif collection.silo.fleet = fleet;\n\n# Describes the policy for creating and managing Silo identity providers\nresource SiloIdentityProviderList {\n\tpermissions = [ "list_children", "create_child" ];\n\n\trelations = { parent_silo: Silo, parent_fleet: Fleet };\n\n\t# Everyone who can read the Silo (which includes all the users in the\n\t# Silo) can see the identity providers in it.\n\t"list_children" if "read" on "parent_silo";\n\n\t# Fleet and Silo administrators can manage the Silo's identity provider\n\t# configuration. This is one of the only areas of Silo configuration\n\t# that Fleet Administrators have permissions on. This is also one of\n\t# the only cases where we need to look two levels up the hierarchy to\n\t# see if somebody has the right permission. For most other things,\n\t# permissions cascade down the hierarchy so we only need to look at the\n\t# parent.\n\t"create_child" if "admin" on "parent_silo";\n\t"create_child" if "admin" on "parent_fleet";\n}\nhas_relation(silo: Silo, "parent_silo", collection: SiloIdentityProviderList)\n\tif collection.silo = silo;\nhas_relation(fleet: Fleet, "parent_fleet", collection: SiloIdentityProviderList)\n\tif collection.silo.fleet = fleet;\n\n# Describes the policy for creating and managing Silo users (mostly intended for\n# API-managed users)\nresource SiloUserList {\n\tpermissions = [ "list_children", "create_child" ];\n\n\trelations = { parent_silo: Silo, parent_fleet: Fleet };\n\n\t# Everyone who can read the Silo (which includes all the users in the\n\t# Silo) can see the users in it.\n\t"list_children" if "read" on "parent_silo";\n\n\t# Fleet and Silo administrators can manage the Silo's users. This is\n\t# one of the only areas of Silo configuration that Fleet Administrators\n\t# have permissions on. This is also one of the few cases (so far) where\n\t# we need to look two levels up the hierarchy to see if somebody has the\n\t# right permission. For most other things, permissions cascade down the\n\t# hierarchy so we only need to look at the parent.\n\t"create_child" if "admin" on "parent_silo";\n\t"list_children" if "admin" on "parent_fleet";\n\t"create_child" if "admin" on "parent_fleet";\n}\nhas_relation(silo: Silo, "parent_silo", collection: SiloUserList)\n\tif collection.silo = silo;\nhas_relation(fleet: Fleet, "parent_fleet", collection: SiloUserList)\n\tif collection.silo.fleet = fleet;\n\n# These rules grants the external authenticator role the permissions it needs to\n# read silo users and modify their sessions. This is necessary for login to\n# work.\nhas_permission(actor: AuthenticatedActor, "read", silo: Silo)\n\tif has_role(actor, "external-authenticator", silo.fleet);\nhas_permission(actor: AuthenticatedActor, "read", user: SiloUser)\n\tif has_role(actor, "external-authenticator", user.silo.fleet);\nhas_permission(actor: AuthenticatedActor, "modify", user: SiloUser)\n\tif has_role(actor, "external-authenticator", user.silo.fleet);\nhas_permission(actor: AuthenticatedActor, "read", group: SiloGroup)\n\tif has_role(actor, "external-authenticator", group.silo.fleet);\nhas_permission(actor: AuthenticatedActor, "modify", group: SiloGroup)\n\tif has_role(actor, "external-authenticator", group.silo.fleet);\n\nhas_permission(actor: AuthenticatedActor, "read", session: ConsoleSession)\n\tif has_role(actor, "external-authenticator", session.fleet);\nhas_permission(actor: AuthenticatedActor, "modify", session: ConsoleSession)\n\tif has_role(actor, "external-authenticator", session.fleet);\n\n# All authenticated users can read and delete device authn requests because\n# by necessity these operations happen before we've figured out what user (or\n# even Silo) the device auth is associated with. Any user can claim a device\n# auth request with the right user code (that's how it works) -- it's the user\n# code and associated logic that prevents unauthorized access here.\nhas_permission(_actor: AuthenticatedActor, "read", _device_auth: DeviceAuthRequest);\nhas_permission(_actor: AuthenticatedActor, "modify", _device_auth: DeviceAuthRequest);\n\nhas_permission(actor: AuthenticatedActor, "read", device_token: DeviceAccessToken)\n\tif has_role(actor, "external-authenticator", device_token.fleet);\n\nhas_permission(actor: AuthenticatedActor, "read", identity_provider: IdentityProvider)\n\tif has_role(actor, "external-authenticator", identity_provider.silo.fleet);\n\nhas_permission(actor: AuthenticatedActor, "read", saml_identity_provider: SamlIdentityProvider)\n\tif has_role(actor, "external-authenticator", saml_identity_provider.silo.fleet);\n\n# Describes the policy for who can access the internal database.\nresource Database {\n\tpermissions = [\n\t # "query" is required to perform any query against the database,\n\t # whether a read or write query. This is checked when an operation\n\t # checks out a database connection from the connection pool.\n\t #\n\t # Any authenticated user gets this permission. There's generally\n\t # some other authz check involved in the database query. For\n\t # example, if you're querying the database to "read" a "Project", we\n\t # should also be checking that. So why do we do this at all? It's\n\t # a belt-and-suspenders measure so that if we somehow introduced an\n\t # unauthenticated code path that hits the database, it cannot be\n\t # used to DoS the database because we won't allow the operation to\n\t # make the query. (As long as the code path _is_ authenticated, we\n\t # can use throttling mechanisms to prevent DoS.)\n\t "query",\n\n\t # "modify" is required to populate database data that's delivered\n\t # with the system. It should also be required for schema changes,\n\t # when we support those. This is separate from "query" so that we\n\t # cannot accidentally invoke these code paths from API calls and\n\t # other general functions.\n\t "modify"\n\t];\n}\n\n# All authenticated users have the "query" permission on the database.\nhas_permission(_actor: AuthenticatedActor, "query", _resource: Database);\n\n# The "db-init" user is the only one with the "modify" permission.\nhas_permission(USER_DB_INIT: AuthenticatedActor, "modify", _resource: Database);\nhas_permission(USER_DB_INIT: AuthenticatedActor, "create_child", _resource: IpPoolList);\n# It also has "admin" on the internal silo to populate it with built-in resources.\n# TODO-completeness: actually limit to just internal silo and not all silos\nhas_role(USER_DB_INIT: AuthenticatedActor, "admin", _silo: Silo);\n\n# Allow the internal API admin permissions on all silos.\nhas_role(USER_INTERNAL_API: AuthenticatedActor, "admin", _silo: Silo);\n\n\n\n resource Disk {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_project: Project };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(parent: Project, "containing_project", child: Disk)\n if child.project = parent;\n \n\n resource Snapshot {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_project: Project };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(parent: Project, "containing_project", child: Snapshot)\n if child.project = parent;\n \n\n resource ProjectImage {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_project: Project };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(parent: Project, "containing_project", child: ProjectImage)\n if child.project = parent;\n \n\n resource Instance {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_project: Project };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(parent: Project, "containing_project", child: Instance)\n if child.project = parent;\n \n\n resource IpPool {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: IpPool)\n if child.fleet = fleet;\n \n\n resource InstanceNetworkInterface {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = {\n containing_project: Project,\n parent: Instance\n };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(project: Project, "containing_project", child: InstanceNetworkInterface)\n if has_relation(project, "containing_project", child.instance);\n\n has_relation(parent: Instance, "parent", child: InstanceNetworkInterface)\n if child.instance = parent;\n \n\n resource Vpc {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_project: Project };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(parent: Project, "containing_project", child: Vpc)\n if child.project = parent;\n \n\n resource VpcRouter {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = {\n containing_project: Project,\n parent: Vpc\n };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(project: Project, "containing_project", child: VpcRouter)\n if has_relation(project, "containing_project", child.vpc);\n\n has_relation(parent: Vpc, "parent", child: VpcRouter)\n if child.vpc = parent;\n \n\n resource RouterRoute {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = {\n containing_project: Project,\n parent: VpcRouter\n };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(project: Project, "containing_project", child: RouterRoute)\n if has_relation(project, "containing_project", child.vpc_router);\n\n has_relation(parent: VpcRouter, "parent", child: RouterRoute)\n if child.vpc_router = parent;\n \n\n resource VpcSubnet {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = {\n containing_project: Project,\n parent: Vpc\n };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(project: Project, "containing_project", child: VpcSubnet)\n if has_relation(project, "containing_project", child.vpc);\n\n has_relation(parent: Vpc, "parent", child: VpcSubnet)\n if child.vpc = parent;\n \n\n resource FloatingIp {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_project: Project };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(parent: Project, "containing_project", child: FloatingIp)\n if child.project = parent;\n \n\n resource Image {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_silo: Silo };\n "list_children" if "viewer" on "containing_silo";\n "read" if "viewer" on "containing_silo";\n "modify" if "collaborator" on "containing_silo";\n "create_child" if "collaborator" on "containing_silo";\n }\n\n has_relation(parent: Silo, "containing_silo", child: Image)\n if child.silo = parent;\n \n\n resource SiloImage {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_silo: Silo };\n "list_children" if "viewer" on "containing_silo";\n "read" if "viewer" on "containing_silo";\n "modify" if "collaborator" on "containing_silo";\n "create_child" if "collaborator" on "containing_silo";\n }\n\n has_relation(parent: Silo, "containing_silo", child: SiloImage)\n if child.silo = parent;\n \n\n resource AddressLot {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: AddressLot)\n if child.fleet = fleet;\n \n\n resource Blueprint {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: Blueprint)\n if child.fleet = fleet;\n \n\n resource LoopbackAddress {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: LoopbackAddress)\n if child.fleet = fleet;\n \n\n\n resource ConsoleSession {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: ConsoleSession)\n if child.fleet = fleet;\n \n\n resource DeviceAuthRequest {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: DeviceAuthRequest)\n if child.fleet = fleet;\n \n\n resource DeviceAccessToken {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: DeviceAccessToken)\n if child.fleet = fleet;\n \n\n resource PhysicalDisk {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: PhysicalDisk)\n if child.fleet = fleet;\n \n\n resource Rack {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: Rack)\n if child.fleet = fleet;\n \n\n resource RoleBuiltin {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: RoleBuiltin)\n if child.fleet = fleet;\n \n\n\n\n\n\n\n\n resource Sled {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: Sled)\n if child.fleet = fleet;\n \n\n resource TufRepo {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: TufRepo)\n if child.fleet = fleet;\n \n\n resource TufArtifact {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: TufArtifact)\n if child.fleet = fleet;\n \n\n resource Zpool {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: Zpool)\n if child.fleet = fleet;\n \n\n resource Service {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: Service)\n if child.fleet = fleet;\n \n\n resource UserBuiltin {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: UserBuiltin)\n if child.fleet = fleet;\n file = nexus/auth/src/authz/oso_generic.rs:89
602024-08-22T17:49:32.512ZINFOnexus (ServerContext): Setting up resolver using DNS servers for subnet: Ipv6Subnet { net: Ipv6Net { addr: fd00:1122:3344::, width: 48 } } file = nexus/src/context.rs:214
612024-08-22T17:49:32.516ZINFOnexus (DnsResolver): new DNS resolver addresses = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53, [fd00:1122:3344:4::1]:53, [fd00:1122:3344:5::1]:53] file = internal-dns/src/resolver.rs:73
622024-08-22T17:49:32.519ZINFOnexus (ServerContext): Setting up qorb pool from DNS dns_addrs = [\n [fd00:1122:3344:1::1]:53,\n [fd00:1122:3344:2::1]:53,\n [fd00:1122:3344:3::1]:53,\n [fd00:1122:3344:4::1]:53,\n [fd00:1122:3344:5::1]:53,\n] file = nexus/src/context.rs:263
632024-08-22T17:49:42.621ZINFOnexus: Database schema version is up to date desired_version = 90.0.0 file = nexus/db-queries/src/db/datastore/db_metadata.rs:145 found_version = 90.0.0
642024-08-22T17:49:42.623ZINFOnexus (SEC): SEC running file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/steno-0.4.1/src/sec.rs:813 sec_id = eaa260db-2b19-4ae1-97e8-5735c9f5d955
652024-08-22T17:49:42.626ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-c961ede4-7532-4ca1-95a5-690b4d7c9b5a.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-c961ede4-7532-4ca1-95a5-690b4d7c9b5a.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1571, tv_nsec: 159658540 } })
662024-08-22T17:49:42.629ZINFOnexus (Nexus): Determining switch slots managed by switch zones file = nexus/src/app/mod.rs:1050
672024-08-22T17:49:42.680ZINFOnexus (Nexus): determining switch slot managed by dendrite zone file = nexus/src/app/mod.rs:1058 zone_address = fd00:1122:3344:101::2
682024-08-22T17:49:42.681ZDEBGnexus (MgsClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
692024-08-22T17:49:42.688ZDEBGnexus (MgsClient): client response result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "4ecd3059-adab-47ea-99da-5c5327e0f687", "content-length": "26", "date": "Thu, 22 Aug 2024 17:49:42 GMT"} })
702024-08-22T17:49:42.688ZINFOnexus (Nexus): identified switch slot for dendrite zone file = nexus/src/app/mod.rs:1063 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
712024-08-22T17:49:42.688ZINFOnexus (Nexus): completed mapping dendrite zones to switch slots file = nexus/src/app/mod.rs:1095 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
722024-08-22T17:49:42.720ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-c961ede4-7532-4ca1-95a5-690b4d7c9b5a.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-c961ede4-7532-4ca1-95a5-690b4d7c9b5a.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1571, tv_nsec: 256634161 } })
732024-08-22T17:49:42.721ZINFOnexus (Nexus): Determining switch slots managed by switch zones file = nexus/src/app/mod.rs:1050
742024-08-22T17:49:42.754ZINFOnexus (Nexus): determining switch slot managed by dendrite zone file = nexus/src/app/mod.rs:1058 zone_address = fd00:1122:3344:101::2
752024-08-22T17:49:42.754ZDEBGnexus (MgsClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
762024-08-22T17:49:42.755ZDEBGnexus (MgsClient): client response result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "7291c783-32a7-4f43-911d-b8382de256d7", "content-length": "26", "date": "Thu, 22 Aug 2024 17:49:42 GMT"} })
772024-08-22T17:49:42.755ZINFOnexus (Nexus): identified switch slot for dendrite zone file = nexus/src/app/mod.rs:1063 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
782024-08-22T17:49:42.755ZINFOnexus (Nexus): completed mapping dendrite zones to switch slots file = nexus/src/app/mod.rs:1095 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
792024-08-22T17:49:42.837ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
802024-08-22T17:49:42.849ZDEBGnexus (DataLoader): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
812024-08-22T17:49:42.852ZDEBGnexus (DataLoader): attempting to create built-in users
822024-08-22T17:49:42.852ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
832024-08-22T17:49:42.852ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
842024-08-22T17:49:42.871ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = GET path = /artifacts/{kind}/{name}/{version}
852024-08-22T17:49:42.871ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = GET path = /bgtasks
862024-08-22T17:49:42.872ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = POST path = /bgtasks/activate
872024-08-22T17:49:42.872ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = GET path = /bgtasks/view/{bgtask_name}
882024-08-22T17:49:42.872ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = POST path = /crucible/0/upstairs/{upstairs_id}/downstairs/{downstairs_id}/stop-request
892024-08-22T17:49:42.872ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = POST path = /crucible/0/upstairs/{upstairs_id}/downstairs/{downstairs_id}/stopped
902024-08-22T17:49:42.872ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = POST path = /crucible/0/upstairs/{upstairs_id}/repair/{repair_id}/progress
912024-08-22T17:49:42.872ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = POST path = /crucible/0/upstairs/{upstairs_id}/repair-finish
922024-08-22T17:49:42.872ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = POST path = /crucible/0/upstairs/{upstairs_id}/repair-start
932024-08-22T17:49:42.872ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = POST path = /demo-saga
942024-08-22T17:49:42.872ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = POST path = /demo-saga/{demo_saga_id}/complete
952024-08-22T17:49:42.872ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = GET path = /deployment/blueprints/all
962024-08-22T17:49:42.872ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = DELETE path = /deployment/blueprints/all/{blueprint_id}
972024-08-22T17:49:42.872ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = GET path = /deployment/blueprints/all/{blueprint_id}
982024-08-22T17:49:42.872ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = POST path = /deployment/blueprints/import
992024-08-22T17:49:42.872ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = POST path = /deployment/blueprints/regenerate
1002024-08-22T17:49:42.872ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = GET path = /deployment/blueprints/target
1012024-08-22T17:49:42.872ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = POST path = /deployment/blueprints/target
1022024-08-22T17:49:42.872ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = PUT path = /deployment/blueprints/target/enabled
1032024-08-22T17:49:42.872ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = POST path = /disk/{disk_id}/remove-read-only-parent
1042024-08-22T17:49:42.872ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = PUT path = /disks/{disk_id}
1052024-08-22T17:49:42.872ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = PUT path = /instances/{instance_id}
1062024-08-22T17:49:42.872ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = POST path = /instances/{instance_id}/migrate
1072024-08-22T17:49:42.872ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = POST path = /metrics/collectors
1082024-08-22T17:49:42.872ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = GET path = /metrics/collectors/{collector_id}/producers
1092024-08-22T17:49:42.872ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = POST path = /metrics/producers
1102024-08-22T17:49:42.872ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = GET path = /nat/ipv4/changeset/{from_gen}
1112024-08-22T17:49:42.872ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = POST path = /physical-disk/expunge
1122024-08-22T17:49:42.872ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = GET path = /probes/{sled}
1132024-08-22T17:49:42.872ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = PUT path = /racks/{rack_id}/initialization-complete
1142024-08-22T17:49:42.872ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = GET path = /sagas
1152024-08-22T17:49:42.873ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = GET path = /sagas/{saga_id}
1162024-08-22T17:49:42.873ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = GET path = /sled-agents/{sled_id}
1172024-08-22T17:49:42.873ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = POST path = /sled-agents/{sled_id}
1182024-08-22T17:49:42.873ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = POST path = /sled-agents/{sled_id}/firewall-rules-update
1192024-08-22T17:49:42.873ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = POST path = /sleds/add
1202024-08-22T17:49:42.873ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = POST path = /sleds/expunge
1212024-08-22T17:49:42.873ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = GET path = /sleds/uninitialized
1222024-08-22T17:49:42.873ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = PUT path = /switch/{switch_id}
1232024-08-22T17:49:42.873ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = POST path = /volume/{volume_id}/remove-read-only-parent
1242024-08-22T17:49:42.873ZINFOnexus (dropshot_internal): listening file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:205 local_addr = [fd00:1122:3344:101::c]:12221
1252024-08-22T17:49:42.873ZDEBGnexus (dropshot_internal): successfully registered DTrace USDT probes local_addr = [fd00:1122:3344:101::c]:12221
1262024-08-22T17:49:42.875ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
1272024-08-22T17:49:42.876ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
1282024-08-22T17:49:42.921ZINFOnexus (DataLoader): created 5 built-in users file = nexus/db-queries/src/db/datastore/silo_user.rs:394
1292024-08-22T17:49:42.921ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
1302024-08-22T17:49:42.922ZDEBGnexus (DataLoader): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1312024-08-22T17:49:42.922ZDEBGnexus (DataLoader): attempting to create built-in roles
1322024-08-22T17:49:42.922ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
1332024-08-22T17:49:42.922ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1342024-08-22T17:49:42.938ZINFOnexus (DataLoader): created 10 built-in roles file = nexus/db-queries/src/db/datastore/role.rs:85
1352024-08-22T17:49:42.938ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
1362024-08-22T17:49:42.939ZDEBGnexus (DataLoader): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1372024-08-22T17:49:42.939ZDEBGnexus (DataLoader): attempting to create built-in role assignments
1382024-08-22T17:49:42.939ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
1392024-08-22T17:49:42.939ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1402024-08-22T17:49:42.981ZWARNnexus: Cannot look up rack: Object (of type ById(c66d93a6-e2c3-4b19-9564-3b58bad50561)) not found: rack file = nexus/src/app/rack.rs:771
1412024-08-22T17:49:43.052ZINFOnexus (DataLoader): created 4 built-in role assignments file = nexus/db-queries/src/db/datastore/role.rs:114
1422024-08-22T17:49:43.052ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
1432024-08-22T17:49:43.054ZDEBGnexus (DataLoader): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1442024-08-22T17:49:43.054ZDEBGnexus (DataLoader): attempting to create built-in silos
1452024-08-22T17:49:43.054ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
1462024-08-22T17:49:43.054ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1472024-08-22T17:49:43.121ZINFOnexus (DataLoader): created 2 built-in silos file = nexus/db-queries/src/db/datastore/silo.rs:95
1482024-08-22T17:49:43.121ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
1492024-08-22T17:49:43.122ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1502024-08-22T17:49:43.128ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
1512024-08-22T17:49:43.129ZDEBGnexus (DataLoader): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1522024-08-22T17:49:43.129ZDEBGnexus (DataLoader): attempting to create built-in projects
1532024-08-22T17:49:43.129ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
1542024-08-22T17:49:43.129ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1552024-08-22T17:49:43.172ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
1562024-08-22T17:49:43.173ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1572024-08-22T17:49:43.521ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
1582024-08-22T17:49:43.521ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1592024-08-22T17:49:43.660ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
1602024-08-22T17:49:43.663ZDEBGnexus (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) } result = Ok(())
1612024-08-22T17:49:43.663ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
1622024-08-22T17:49:43.663ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1632024-08-22T17:49:44.092ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
1642024-08-22T17:49:44.093ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1652024-08-22T17:49:44.094ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
1662024-08-22T17:49:44.096ZDEBGnexus (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) } result = Ok(())
1672024-08-22T17:49:44.096ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
1682024-08-22T17:49:44.096ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1692024-08-22T17:49:44.121ZINFOnexus (DataLoader): created built-in services project file = nexus/db-queries/src/db/datastore/project.rs:122
1702024-08-22T17:49:44.121ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
1712024-08-22T17:49:44.122ZDEBGnexus (DataLoader): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1722024-08-22T17:49:44.122ZDEBGnexus (DataLoader): attempting to create built-in VPCs
1732024-08-22T17:49:44.122ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
1742024-08-22T17:49:44.122ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1752024-08-22T17:49:44.133ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
1762024-08-22T17:49:44.133ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1772024-08-22T17:49:44.151ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
1782024-08-22T17:49:44.151ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1792024-08-22T17:49:44.153ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
1802024-08-22T17:49:44.153ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1812024-08-22T17:49:44.156ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
1822024-08-22T17:49:44.156ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1832024-08-22T17:49:44.317ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
1842024-08-22T17:49:44.319ZDEBGnexus (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) } result = Ok(())
1852024-08-22T17:49:44.319ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
1862024-08-22T17:49:44.319ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1872024-08-22T17:49:44.321ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
1882024-08-22T17:49:44.321ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1892024-08-22T17:49:44.323ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
1902024-08-22T17:49:44.323ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1912024-08-22T17:49:44.324ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
1922024-08-22T17:49:44.326ZDEBGnexus (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) } result = Ok(())
1932024-08-22T17:49:44.326ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
1942024-08-22T17:49:44.327ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1952024-08-22T17:49:44.381ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
1962024-08-22T17:49:44.382ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1972024-08-22T17:49:44.391ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
1982024-08-22T17:49:44.391ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1992024-08-22T17:49:44.392ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2002024-08-22T17:49:44.393ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2012024-08-22T17:49:44.394ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2022024-08-22T17:49:44.394ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2032024-08-22T17:49:44.395ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2042024-08-22T17:49:44.398ZDEBGnexus (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") } result = Ok(())
2052024-08-22T17:49:44.398ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2062024-08-22T17:49:44.398ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2072024-08-22T17:49:44.439ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2082024-08-22T17:49:44.439ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2092024-08-22T17:49:44.440ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2102024-08-22T17:49:44.441ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2112024-08-22T17:49:44.442ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2122024-08-22T17:49:44.442ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2132024-08-22T17:49:44.444ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2142024-08-22T17:49:44.447ZDEBGnexus (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = VpcRouter { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-074c-4000-8000-000000000001, lookup_type: ById(001de000-074c-4000-8000-000000000001) } result = Ok(())
2152024-08-22T17:49:44.447ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2162024-08-22T17:49:44.447ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2172024-08-22T17:49:44.494ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2182024-08-22T17:49:44.495ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2192024-08-22T17:49:44.533ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2202024-08-22T17:49:44.533ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2212024-08-22T17:49:44.539ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2222024-08-22T17:49:44.539ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2232024-08-22T17:49:44.540ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2242024-08-22T17:49:44.543ZDEBGnexus (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = VpcRouter { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-074c-4000-8000-000000000001, lookup_type: ById(001de000-074c-4000-8000-000000000001) } result = Ok(())
2252024-08-22T17:49:44.543ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2262024-08-22T17:49:44.544ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2272024-08-22T17:49:44.611ZDEBGnexus (DataLoader): attempting to create built-in VPC firewall rules
2282024-08-22T17:49:44.612ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2292024-08-22T17:49:44.612ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2302024-08-22T17:49:44.615ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2312024-08-22T17:49:44.615ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2322024-08-22T17:49:44.616ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2332024-08-22T17:49:44.617ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2342024-08-22T17:49:44.618ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2352024-08-22T17:49:44.618ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2362024-08-22T17:49:44.620ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2372024-08-22T17:49:44.620ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2382024-08-22T17:49:44.621ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2392024-08-22T17:49:44.621ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2402024-08-22T17:49:44.623ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2412024-08-22T17:49:44.625ZDEBGnexus (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
2422024-08-22T17:49:44.625ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2432024-08-22T17:49:44.625ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2442024-08-22T17:49:44.627ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2452024-08-22T17:49:44.627ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2462024-08-22T17:49:44.630ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2472024-08-22T17:49:44.631ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2482024-08-22T17:49:44.632ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2492024-08-22T17:49:44.635ZDEBGnexus (DataLoader): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
2502024-08-22T17:49:44.635ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2512024-08-22T17:49:44.635ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2522024-08-22T17:49:44.660ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2532024-08-22T17:49:44.660ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2542024-08-22T17:49:44.661ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2552024-08-22T17:49:44.661ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2562024-08-22T17:49:44.662ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2572024-08-22T17:49:44.662ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2582024-08-22T17:49:44.664ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2592024-08-22T17:49:44.666ZDEBGnexus (DataLoader): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
2602024-08-22T17:49:44.666ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2612024-08-22T17:49:44.666ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2622024-08-22T17:49:44.755ZINFOnexus (DataLoader): created built-in VPC firewall rules file = nexus/db-queries/src/db/datastore/vpc.rs:252
2632024-08-22T17:49:44.755ZDEBGnexus (DataLoader): attempting to create built-in VPC Subnets
2642024-08-22T17:49:44.755ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2652024-08-22T17:49:44.756ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2662024-08-22T17:49:44.766ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2672024-08-22T17:49:44.767ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2682024-08-22T17:49:44.768ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2692024-08-22T17:49:44.768ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2702024-08-22T17:49:44.770ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2712024-08-22T17:49:44.770ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2722024-08-22T17:49:44.772ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2732024-08-22T17:49:44.772ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2742024-08-22T17:49:44.774ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2752024-08-22T17:49:44.774ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2762024-08-22T17:49:44.781ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2772024-08-22T17:49:44.783ZDEBGnexus (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
2782024-08-22T17:49:44.783ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2792024-08-22T17:49:44.784ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2802024-08-22T17:49:44.794ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2812024-08-22T17:49:44.794ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2822024-08-22T17:49:44.796ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2832024-08-22T17:49:44.796ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2842024-08-22T17:49:44.797ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2852024-08-22T17:49:44.797ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2862024-08-22T17:49:44.799ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2872024-08-22T17:49:44.801ZDEBGnexus (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
2882024-08-22T17:49:44.828ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2892024-08-22T17:49:44.828ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2902024-08-22T17:49:44.872ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2912024-08-22T17:49:44.872ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2922024-08-22T17:49:44.902ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2932024-08-22T17:49:44.903ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2942024-08-22T17:49:44.904ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2952024-08-22T17:49:44.904ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2962024-08-22T17:49:44.906ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2972024-08-22T17:49:44.906ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2982024-08-22T17:49:44.907ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2992024-08-22T17:49:44.910ZDEBGnexus (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = VpcRouter { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-074c-4000-8000-000000000001, lookup_type: ById(001de000-074c-4000-8000-000000000001) } result = Ok(())
3002024-08-22T17:49:44.910ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3012024-08-22T17:49:44.911ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3022024-08-22T17:49:44.931ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3032024-08-22T17:49:44.932ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3042024-08-22T17:49:44.942ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3052024-08-22T17:49:44.943ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3062024-08-22T17:49:44.944ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3072024-08-22T17:49:44.944ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3082024-08-22T17:49:44.946ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3092024-08-22T17:49:44.946ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3102024-08-22T17:49:44.947ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3112024-08-22T17:49:44.950ZDEBGnexus (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
3122024-08-22T17:49:44.965ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3132024-08-22T17:49:44.966ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3142024-08-22T17:49:44.982ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
3152024-08-22T17:49:44.982ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
3162024-08-22T17:49:44.991ZWARNnexus: Cannot look up rack: Object (of type ById(c66d93a6-e2c3-4b19-9564-3b58bad50561)) not found: rack file = nexus/src/app/rack.rs:771
3172024-08-22T17:49:45.060ZERROnexus (DataLoader): gave up trying to populate built-in PopulateBuiltinVpcs error_message = InternalError { internal_message: "unexpected database error: Record not found" } file = nexus/src/populate.rs:126
3182024-08-22T17:49:45.060ZERROnexus: populate failed file = nexus/src/app/mod.rs:516
3192024-08-22T17:49:45.060ZINFOnexus: activating background tasks file = nexus/src/app/mod.rs:522
3202024-08-22T17:49:45.065ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 1 reason = Timeout
3212024-08-22T17:49:45.065ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 1 reason = Timeout
3222024-08-22T17:49:45.065ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 1 reason = Timeout
3232024-08-22T17:49:45.065ZDEBGnexus (BackgroundTasks): activating background_task = service_firewall_rule_propagation iteration = 1 reason = Timeout
3242024-08-22T17:49:45.065ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 1 reason = Timeout
3252024-08-22T17:49:45.065ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 1 reason = Timeout
3262024-08-22T17:49:45.065ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 1 reason = Timeout
3272024-08-22T17:49:45.065ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 1 reason = Timeout
3282024-08-22T17:49:45.065ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 1 reason = Timeout
3292024-08-22T17:49:45.065ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
3302024-08-22T17:49:45.065ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
3312024-08-22T17:49:45.065ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
3322024-08-22T17:49:45.066ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
3332024-08-22T17:49:45.066ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:56
3342024-08-22T17:49:45.066ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 1 reason = Timeout
3352024-08-22T17:49:45.066ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 1 reason = Timeout
3362024-08-22T17:49:45.066ZWARNnexus (BackgroundTasks): DNS propagation: skipped background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:61 reason = no config nor servers
3372024-08-22T17:49:45.066ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 1 reason = Timeout
3382024-08-22T17:49:45.066ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
3392024-08-22T17:49:45.066ZDEBGnexus (BackgroundTasks): activating background_task = crdb_node_id_collector iteration = 1 reason = Timeout
3402024-08-22T17:49:45.066ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = crdb_node_id_collector file = nexus/src/app/background/tasks/crdb_node_id_collector.rs:78 reason = no blueprint
3412024-08-22T17:49:45.066ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
3422024-08-22T17:49:45.066ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
3432024-08-22T17:49:45.066ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
3442024-08-22T17:49:45.067ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
3452024-08-22T17:49:45.067ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 679.234\u{b5}s iteration = 1 status = {"error":"no config nor servers"}
3462024-08-22T17:49:45.067ZDEBGnexus (BackgroundTasks): activation complete background_task = crdb_node_id_collector elapsed = 634.063\u{b5}s iteration = 1 status = {"error":"no blueprint"}
3472024-08-22T17:49:45.067ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 1 reason = Timeout
3482024-08-22T17:49:45.067ZDEBGnexus (BackgroundTasks): activating background_task = inventory_collection iteration = 1 reason = Timeout
3492024-08-22T17:49:45.067ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 1 reason = Timeout
3502024-08-22T17:49:45.067ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 1 reason = Timeout
3512024-08-22T17:49:45.067ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 1 reason = Timeout
3522024-08-22T17:49:45.067ZWARNnexus (BackgroundTasks): DNS propagation: skipped background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:61 reason = no config nor servers
3532024-08-22T17:49:45.067ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 1 reason = Timeout
3542024-08-22T17:49:45.067ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 1.481048ms iteration = 1 status = {"error":"no config nor servers"}
3552024-08-22T17:49:45.067ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 1 reason = Timeout
3562024-08-22T17:49:45.067ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 1 reason = Timeout
3572024-08-22T17:49:45.067ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
3582024-08-22T17:49:45.067ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 1 reason = Timeout
3592024-08-22T17:49:45.067ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2024-08-22 17:39:45.067693483 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
3602024-08-22T17:49:45.067ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 1 reason = Timeout
3612024-08-22T17:49:45.067ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
3622024-08-22T17:49:45.067ZDEBGnexus (BackgroundTasks): activating background_task = saga_recovery iteration = 1 reason = Timeout
3632024-08-22T17:49:45.067ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
3642024-08-22T17:49:45.067ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 1 reason = Timeout
3652024-08-22T17:49:45.067ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
3662024-08-22T17:49:45.067ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
3672024-08-22T17:49:45.067ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 1 reason = Timeout
3682024-08-22T17:49:45.067ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 1 reason = Timeout
3692024-08-22T17:49:45.067ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 1 reason = Timeout
3702024-08-22T17:49:45.067ZINFOnexus (BackgroundTasks): Abandoned VMM reaper running background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:76
3712024-08-22T17:49:45.067ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 1 reason = Timeout
3722024-08-22T17:49:45.067ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 1 reason = Timeout
3732024-08-22T17:49:45.067ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 1 reason = Timeout
3742024-08-22T17:49:45.069ZDEBGnexus (SagaRecoverer): listing candidate sagas for recovery
3752024-08-22T17:49:45.070ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
3762024-08-22T17:49:45.070ZDEBGnexus (SagaRecoverer): roles roles = RoleSet { roles: {} }
3772024-08-22T17:49:45.070ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
3782024-08-22T17:49:45.070ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
3792024-08-22T17:49:45.070ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
3802024-08-22T17:49:45.070ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
3812024-08-22T17:49:45.070ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
3822024-08-22T17:49:45.070ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3832024-08-22T17:49:45.070ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
3842024-08-22T17:49:45.071ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
3852024-08-22T17:49:45.071ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
3862024-08-22T17:49:45.071ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
3872024-08-22T17:49:45.071ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
3882024-08-22T17:49:45.071ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:71 reason = no blueprint
3892024-08-22T17:49:45.071ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 5.235387ms iteration = 1 status = {"error":"no blueprint"}
3902024-08-22T17:49:45.072ZDEBGnexus (service-firewall-rule-progator): starting background task for service firewall rule propagation background_task = service_firewall_rule_propagation
3912024-08-22T17:49:45.072ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
3922024-08-22T17:49:45.072ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("9488ce0a-6732-4f75-baf9-4ecfa406df91.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("afac2eab-cadf-48d0-94a7-ad562ade88c6.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("afac2eab-cadf-48d0-94a7-ad562ade88c6.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::9))) }], valid_until: Instant { tv_sec: 1573, tv_nsec: 608448635 } })
3932024-08-22T17:49:45.073ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
3942024-08-22T17:49:45.073ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
3952024-08-22T17:49:45.073ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
3962024-08-22T17:49:45.074ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
3972024-08-22T17:49:45.074ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
3982024-08-22T17:49:45.074ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
3992024-08-22T17:49:45.074ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
4002024-08-22T17:49:45.074ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4012024-08-22T17:49:45.074ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
4022024-08-22T17:49:45.074ZDEBGnexus (SagaRecoverer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000003a8a, .. }) resource = Database result = Ok(())
4032024-08-22T17:49:45.074ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
4042024-08-22T17:49:45.074ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
4052024-08-22T17:49:45.075ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
4062024-08-22T17:49:45.076ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4072024-08-22T17:49:45.076ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("6507c79b-9a18-46c7-930d-c98e8662b87f.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("a09ea923-b85c-428d-b289-6936a0c2542c.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("c0ef020c-52c1-4b9a-a085-445fab31264f.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("c0ef020c-52c1-4b9a-a085-445fab31264f.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:3::1))) }], valid_until: Instant { tv_sec: 1573, tv_nsec: 612055874 } })
4082024-08-22T17:49:45.077ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
4092024-08-22T17:49:45.077ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
4102024-08-22T17:49:45.077ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
4112024-08-22T17:49:45.077ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
4122024-08-22T17:49:45.077ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
4132024-08-22T17:49:45.077ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4142024-08-22T17:49:45.077ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4152024-08-22T17:49:45.077ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
4162024-08-22T17:49:45.078ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
4172024-08-22T17:49:45.078ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
4182024-08-22T17:49:45.078ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
4192024-08-22T17:49:45.078ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
4202024-08-22T17:49:45.078ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
4212024-08-22T17:49:45.081ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
4222024-08-22T17:49:45.081ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
4232024-08-22T17:49:45.081ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
4242024-08-22T17:49:45.081ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
4252024-08-22T17:49:45.084ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 19.199029ms iteration = 1 status = {"errors":[],"found_port_ok":[]}
4262024-08-22T17:49:45.088ZINFOnexus (BackgroundTasks): found DNS servers (initial) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external dns_group = external file = nexus/src/app/background/tasks/dns_servers.rs:113
4272024-08-22T17:49:45.088ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 23.012079ms iteration = 1 status = {"addresses":["[fd00:1122:3344:101::8]:5353","[fd00:1122:3344:101::9]:5353"]}
4282024-08-22T17:49:45.088ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 2 reason = Dependency
4292024-08-22T17:49:45.088ZWARNnexus (BackgroundTasks): DNS propagation: skipped background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:69 reason = no config
4302024-08-22T17:49:45.088ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 5.38\u{b5}s iteration = 2 status = {"error":"no config"}
4312024-08-22T17:49:45.090ZINFOnexus (BackgroundTasks): found DNS servers (initial) addresses = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal dns_group = internal file = nexus/src/app/background/tasks/dns_servers.rs:113
4322024-08-22T17:49:45.090ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 24.528087ms iteration = 1 status = {"addresses":["[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:3::1]:5353"]}
4332024-08-22T17:49:45.090ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 2 reason = Dependency
4342024-08-22T17:49:45.090ZWARNnexus (BackgroundTasks): DNS propagation: skipped background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:69 reason = no config
4352024-08-22T17:49:45.090ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 5.809\u{b5}s iteration = 2 status = {"error":"no config"}
4362024-08-22T17:49:45.094ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
4372024-08-22T17:49:45.094ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4382024-08-22T17:49:45.094ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
4392024-08-22T17:49:45.096ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
4402024-08-22T17:49:45.096ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
4412024-08-22T17:49:45.097ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
4422024-08-22T17:49:45.097ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
4432024-08-22T17:49:45.098ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
4442024-08-22T17:49:45.127ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2024-08-22 17:39:45.067693483 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
4452024-08-22T17:49:45.127ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 62.527524ms iteration = 1 status = {"expiration":"2024-08-22T17:39:45.067693483Z","pruned":[]}
4462024-08-22T17:49:45.128ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-c961ede4-7532-4ca1-95a5-690b4d7c9b5a.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-c961ede4-7532-4ca1-95a5-690b4d7c9b5a.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1573, tv_nsec: 664203103 } })
4472024-08-22T17:49:45.129ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1050
4482024-08-22T17:49:45.130ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4492024-08-22T17:49:45.134ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
4502024-08-22T17:49:45.134ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
4512024-08-22T17:49:45.134ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
4522024-08-22T17:49:45.136ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
4532024-08-22T17:49:45.136ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
4542024-08-22T17:49:45.139ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-c961ede4-7532-4ca1-95a5-690b4d7c9b5a.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-c961ede4-7532-4ca1-95a5-690b4d7c9b5a.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1573, tv_nsec: 675151940 } })
4552024-08-22T17:49:45.140ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1050
4562024-08-22T17:49:45.140ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4572024-08-22T17:49:45.144ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
4582024-08-22T17:49:45.169ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4592024-08-22T17:49:45.172ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
4602024-08-22T17:49:45.172ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4612024-08-22T17:49:45.172ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4622024-08-22T17:49:45.175ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
4632024-08-22T17:49:45.175ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
4642024-08-22T17:49:45.176ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4652024-08-22T17:49:45.177ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4662024-08-22T17:49:45.177ZWARNnexus (BackgroundTasks): failed to read target blueprint background_task = blueprint_loader error = Internal Error: no target blueprint set file = nexus/src/app/background/tasks/blueprint_load.rs:69
4672024-08-22T17:49:45.177ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 112.323531ms iteration = 1 status = {"error":"failed to read target blueprint: Internal Error: no target blueprint set"}
4682024-08-22T17:49:45.179ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
4692024-08-22T17:49:45.179ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
4702024-08-22T17:49:45.179ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
4712024-08-22T17:49:45.180ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
4722024-08-22T17:49:45.180ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
4732024-08-22T17:49:45.180ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
4742024-08-22T17:49:45.181ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
4752024-08-22T17:49:45.181ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
4762024-08-22T17:49:45.182ZWARNnexus (BackgroundTasks): inventory collection is None background_task = service_zone_nat_tracker file = nexus/src/app/background/tasks/sync_service_zone_nat.rs:95
4772024-08-22T17:49:45.182ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 117.137706ms iteration = 1 status = {"error":"inventory collection is None"}
4782024-08-22T17:49:45.182ZINFOnexus (BackgroundTasks): Physical Disk Adoption: Rack not yet initialized background_task = physical_disk_adoption file = nexus/src/app/background/tasks/physical_disk_adoption.rs:79 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561
4792024-08-22T17:49:45.182ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 116.224991ms iteration = 1 status = {"error":"rack not yet initialized: c66d93a6-e2c3-4b19-9564-3b58bad50561"}
4802024-08-22T17:49:45.182ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4812024-08-22T17:49:45.183ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
4822024-08-22T17:49:45.183ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
4832024-08-22T17:49:45.185ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
4842024-08-22T17:49:45.185ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
4852024-08-22T17:49:45.185ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
4862024-08-22T17:49:45.187ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4872024-08-22T17:49:45.189ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 124.271622ms iteration = 1 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
4882024-08-22T17:49:45.191ZINFOnexus (SagaRecoverer): listed in-progress sagas count = 0 file = nexus/src/app/background/tasks/saga_recovery.rs:438
4892024-08-22T17:49:45.191ZDEBGnexus (BackgroundTasks): activation complete background_task = saga_recovery elapsed = 126.220152ms iteration = 1 status = {"last_pass":{"Success":{"nfailed":0,"nfound":0,"nrecovered":0,"nremoved":0,"nskipped":0}},"ntotal_failures":0,"ntotal_finished":0,"ntotal_recovered":0,"ntotal_sec_errors_bad_state":0,"ntotal_sec_errors_missing":0,"ntotal_started":0,"recent_failures":[],"recent_recoveries":[]}
4902024-08-22T17:49:45.192ZWARNnexus (BackgroundTasks): failed to read DNS config background_task = dns_config_external dns_group = external error = Internal Error: expected exactly one latest version for DNS group External, found 0 file = nexus/src/app/background/tasks/dns_config.rs:72
4912024-08-22T17:49:45.192ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 127.069477ms iteration = 1 status = {"error":"failed to read DNS config: Internal Error: expected exactly one latest version for DNS group External, found 0"}
4922024-08-22T17:49:45.192ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
4932024-08-22T17:49:45.192ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
4942024-08-22T17:49:45.193ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
4952024-08-22T17:49:45.193ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 125.76307ms iteration = 1 status = {"errors":[],"garbage_collect_requested":[]}
4962024-08-22T17:49:45.195ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
4972024-08-22T17:49:45.196ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
4982024-08-22T17:49:45.200ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4992024-08-22T17:49:45.202ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
5002024-08-22T17:49:45.202ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
5012024-08-22T17:49:45.202ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
5022024-08-22T17:49:45.206ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
5032024-08-22T17:49:45.206ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:140 found = 0 not_ready_to_be_deleted = 0
5042024-08-22T17:49:45.206ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 141.499982ms iteration = 1 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
5052024-08-22T17:49:45.206ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
5062024-08-22T17:49:45.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
5072024-08-22T17:49:45.207ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1058 zone_address = fd00:1122:3344:101::2
5082024-08-22T17:49:45.207ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
5092024-08-22T17:49:45.207ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "fa082997-8562-4db9-b8f3-739f7c3e8bbf", "content-length": "26", "date": "Thu, 22 Aug 2024 17:49:44 GMT"} })
5102024-08-22T17:49:45.208ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5112024-08-22T17:49:45.211ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1063 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
5122024-08-22T17:49:45.212ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
5132024-08-22T17:49:45.212ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
5142024-08-22T17:49:45.212ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
5152024-08-22T17:49:45.213ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1095 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
5162024-08-22T17:49:45.216ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5172024-08-22T17:49:45.217ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
5182024-08-22T17:49:45.218ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
5192024-08-22T17:49:45.218ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1058 zone_address = fd00:1122:3344:101::2
5202024-08-22T17:49:45.218ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
5212024-08-22T17:49:45.218ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "f3d2112d-9ae0-4427-9a0b-23226d16ede4", "content-length": "26", "date": "Thu, 22 Aug 2024 17:49:44 GMT"} })
5222024-08-22T17:49:45.218ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1063 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
5232024-08-22T17:49:45.218ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
5242024-08-22T17:49:45.218ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
5252024-08-22T17:49:45.218ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
5262024-08-22T17:49:45.220ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1095 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
5272024-08-22T17:49:45.225ZDEBGnexus (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = []
5282024-08-22T17:49:45.228ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-c961ede4-7532-4ca1-95a5-690b4d7c9b5a.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-c961ede4-7532-4ca1-95a5-690b4d7c9b5a.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1573, tv_nsec: 763795018 } })
5292024-08-22T17:49:45.230ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:55
5302024-08-22T17:49:45.230ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
5312024-08-22T17:49:45.230ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
5322024-08-22T17:49:45.231ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
5332024-08-22T17:49:45.239ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5342024-08-22T17:49:45.242ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
5352024-08-22T17:49:45.242ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
5362024-08-22T17:49:45.242ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
5372024-08-22T17:49:45.249ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
5382024-08-22T17:49:45.250ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
5392024-08-22T17:49:45.252ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 186.881355ms iteration = 1 status = {}
5402024-08-22T17:49:45.274ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
5412024-08-22T17:49:45.274ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
5422024-08-22T17:49:45.275ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0
5432024-08-22T17:49:45.275ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:189 found = 0 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0
5442024-08-22T17:49:45.275ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 210.197566ms iteration = 1 status = {"error":null,"error_count":0,"found":0,"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0}
5452024-08-22T17:49:45.285ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
5462024-08-22T17:49:45.286ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
5472024-08-22T17:49:45.288ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5482024-08-22T17:49:45.292ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
5492024-08-22T17:49:45.292ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
5502024-08-22T17:49:45.292ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
5512024-08-22T17:49:45.298ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 233.044704ms iteration = 1 status = {}
5522024-08-22T17:49:45.308ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
5532024-08-22T17:49:45.308ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "0abf3de8-131e-47f8-85e7-ac1a53e38796", "content-length": "1", "date": "Thu, 22 Aug 2024 17:49:45 GMT"} })
5542024-08-22T17:49:45.309ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 244.296133ms iteration = 1 status = {}
5552024-08-22T17:49:45.309ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
5562024-08-22T17:49:45.310ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
5572024-08-22T17:49:45.312ZDEBGnexus (BackgroundTasks): begin collection background_task = inventory_collection
5582024-08-22T17:49:45.312ZDEBGnexus (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[fd00:1122:3344:101::2]:12225
5592024-08-22T17:49:45.312ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/ignition
5602024-08-22T17:49:45.314ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 248.915066ms iteration = 1 status = null
5612024-08-22T17:49:45.315ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/ignition", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "2421ffad-36c2-4c51-8e56-2b97c6d0ca8b", "content-length": "476", "date": "Thu, 22 Aug 2024 17:49:44 GMT"} })
5622024-08-22T17:49:45.318ZWARNnexus (BackgroundTasks): failed to read Silo/DNS/TLS configuration background_task = external_endpoints error = Internal Error: expected at least one external DNS zone file = nexus/src/app/background/tasks/external_endpoints.rs:48
5632024-08-22T17:49:45.318ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 252.761716ms iteration = 1 status = {"error":"failed to read Silo/DNS/TLS configuration: Internal Error: expected at least one external DNS zone"}
5642024-08-22T17:49:45.318ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 252.895417ms iteration = 1 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
5652024-08-22T17:49:45.320ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0
5662024-08-22T17:49:45.324ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/sp/switch/0", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "b2c93d49-e2e5-4b75-9a82-60a5b752768b", "content-length": "734", "date": "Thu, 22 Aug 2024 17:49:45 GMT"} })
5672024-08-22T17:49:45.325ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0
5682024-08-22T17:49:45.325ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 260.289145ms iteration = 1 status = {"region_replacement_started_err":0,"region_replacement_started_ok":0}
5692024-08-22T17:49:45.325ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/sp/switch/0/component/sp/caboose", query: Some("firmware_slot=0"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "d6f3b50e-4fd3-4f6d-adee-6dfa4e633eb8", "content-length": "86", "date": "Thu, 22 Aug 2024 17:49:45 GMT"} })
5702024-08-22T17:49:45.325ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1
5712024-08-22T17:49:45.326ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/sp/switch/0/component/sp/caboose", query: Some("firmware_slot=1"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "d861e868-52b6-4fbd-8ea6-c19bfac0e57c", "content-length": "86", "date": "Thu, 22 Aug 2024 17:49:45 GMT"} })
5722024-08-22T17:49:45.326ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0
5732024-08-22T17:49:45.327ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/sp/switch/0/component/rot/caboose", query: Some("firmware_slot=0"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "389aba79-2cd5-4605-916c-c9093b0fc008", "content-length": "80", "date": "Thu, 22 Aug 2024 17:49:45 GMT"} })
5742024-08-22T17:49:45.327ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1
5752024-08-22T17:49:45.327ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/sp/switch/0/component/rot/caboose", query: Some("firmware_slot=1"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "76acd364-05d3-490a-8923-a2c17c028048", "content-length": "80", "date": "Thu, 22 Aug 2024 17:49:45 GMT"} })
5762024-08-22T17:49:45.327ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0
5772024-08-22T17:49:45.328ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/sp/switch/0/component/stage0/caboose", query: Some("firmware_slot=0"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "44500dd3-8f83-4754-a221-fc21bd64806a", "content-length": "88", "date": "Thu, 22 Aug 2024 17:49:45 GMT"} })
5782024-08-22T17:49:45.328ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1
5792024-08-22T17:49:45.329ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/sp/switch/0/component/stage0/caboose", query: Some("firmware_slot=1"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "353ce406-e94e-4092-b344-6416f8e8dffa", "content-length": "88", "date": "Thu, 22 Aug 2024 17:49:45 GMT"} })
5802024-08-22T17:49:45.329ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa
5812024-08-22T17:49:45.329ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/sp/switch/0/component/rot/cmpa", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "5d18ac48-8c32-4a8c-99d2-05cc8df81510", "content-length": "702", "date": "Thu, 22 Aug 2024 17:49:45 GMT"} })
5822024-08-22T17:49:45.329ZWARNnexus (BackgroundTasks): failed to read DNS config background_task = dns_config_internal dns_group = internal error = Internal Error: expected exactly one latest version for DNS group Internal, found 0 file = nexus/src/app/background/tasks/dns_config.rs:72
5832024-08-22T17:49:45.329ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 264.255676ms iteration = 1 status = {"error":"failed to read DNS config: Internal Error: expected exactly one latest version for DNS group Internal, found 0"}
5842024-08-22T17:49:45.330ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
5852024-08-22T17:49:45.330ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/sp/switch/0/component/rot/cfpa", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "73b1aa60-33b6-479b-96c5-6b328b47b835", "content-length": "727", "date": "Thu, 22 Aug 2024 17:49:44 GMT"} })
5862024-08-22T17:49:45.330ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
5872024-08-22T17:49:45.330ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/sp/switch/0/component/rot/cfpa", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "76ee8e7c-f922-4ccf-baeb-01da101a63fd", "content-length": "729", "date": "Thu, 22 Aug 2024 17:49:45 GMT"} })
5882024-08-22T17:49:45.330ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
5892024-08-22T17:49:45.331ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/sp/switch/0/component/rot/cfpa", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "a19c0bdb-781b-460a-a608-8414141c3e63", "content-length": "728", "date": "Thu, 22 Aug 2024 17:49:45 GMT"} })
5902024-08-22T17:49:45.331ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0
5912024-08-22T17:49:45.331ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/sp/sled/0", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "cf5785a4-798e-4aaa-a0f1-a65441bd3efa", "content-length": "722", "date": "Thu, 22 Aug 2024 17:49:45 GMT"} })
5922024-08-22T17:49:45.331ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0
5932024-08-22T17:49:45.332ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/sp/sled/0/component/sp/caboose", query: Some("firmware_slot=0"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "fbb9ff07-703c-4363-95f4-4df56eaac1d4", "content-length": "84", "date": "Thu, 22 Aug 2024 17:49:45 GMT"} })
5942024-08-22T17:49:45.332ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1
5952024-08-22T17:49:45.332ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/sp/sled/0/component/sp/caboose", query: Some("firmware_slot=1"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "5af6ab0b-2970-4387-ba45-c1993e5002f2", "content-length": "84", "date": "Thu, 22 Aug 2024 17:49:45 GMT"} })
5962024-08-22T17:49:45.332ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0
5972024-08-22T17:49:45.332ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/sp/sled/0/component/rot/caboose", query: Some("firmware_slot=0"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "d8e1b4bf-f0ff-40b0-b217-46b926df3d16", "content-length": "82", "date": "Thu, 22 Aug 2024 17:49:45 GMT"} })
5982024-08-22T17:49:45.332ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1
5992024-08-22T17:49:45.332ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/sp/sled/0/component/rot/caboose", query: Some("firmware_slot=1"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "0f05e401-eab5-4648-acbf-aae2a3446497", "content-length": "82", "date": "Thu, 22 Aug 2024 17:49:45 GMT"} })
6002024-08-22T17:49:45.332ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0
6012024-08-22T17:49:45.333ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/sp/sled/0/component/stage0/caboose", query: Some("firmware_slot=0"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "351e7e37-0648-4ea8-914b-4b81e9398f2b", "content-length": "91", "date": "Thu, 22 Aug 2024 17:49:45 GMT"} })
6022024-08-22T17:49:45.333ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1
6032024-08-22T17:49:45.334ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/sp/sled/0/component/stage0/caboose", query: Some("firmware_slot=1"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "4b5f44af-0ad1-4855-a05d-9c7aa1f10b75", "content-length": "91", "date": "Thu, 22 Aug 2024 17:49:45 GMT"} })
6042024-08-22T17:49:45.334ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa
6052024-08-22T17:49:45.334ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/sp/sled/0/component/rot/cmpa", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "70da33d6-ba3c-41ee-ac30-31662906f752", "content-length": "702", "date": "Thu, 22 Aug 2024 17:49:45 GMT"} })
6062024-08-22T17:49:45.334ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
6072024-08-22T17:49:45.334ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/sp/sled/0/component/rot/cfpa", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "106214cf-cca6-4b90-9886-858e957ad3f7", "content-length": "727", "date": "Thu, 22 Aug 2024 17:49:45 GMT"} })
6082024-08-22T17:49:45.334ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
6092024-08-22T17:49:45.335ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/sp/sled/0/component/rot/cfpa", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "82d78a0f-b90b-4872-a731-9295e390c61f", "content-length": "729", "date": "Thu, 22 Aug 2024 17:49:45 GMT"} })
6102024-08-22T17:49:45.335ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
6112024-08-22T17:49:45.335ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/sp/sled/0/component/rot/cfpa", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "5de83a3c-7e66-4ea0-acd5-eb37e165d274", "content-length": "728", "date": "Thu, 22 Aug 2024 17:49:45 GMT"} })
6122024-08-22T17:49:45.335ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
6132024-08-22T17:49:45.335ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
6142024-08-22T17:49:45.337ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:55
6152024-08-22T17:49:45.337ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:220 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
6162024-08-22T17:49:45.337ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 271.792174ms iteration = 1 status = {"destroyed_active_vmms":0,"error":null,"saga_start_failures":0,"sagas_completed":0,"sagas_failed":0,"sagas_started":0,"terminated_active_migrations":0}
6172024-08-22T17:49:45.337ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6182024-08-22T17:49:45.338ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 272.530768ms iteration = 1 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
6192024-08-22T17:49:45.339ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
6202024-08-22T17:49:45.339ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
6212024-08-22T17:49:45.339ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
6222024-08-22T17:49:45.340ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
6232024-08-22T17:49:45.341ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
6242024-08-22T17:49:45.343ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
6252024-08-22T17:49:45.343ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
6262024-08-22T17:49:45.345ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
6272024-08-22T17:49:45.345ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
6282024-08-22T17:49:45.347ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
6292024-08-22T17:49:45.347ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
6302024-08-22T17:49:45.349ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
6312024-08-22T17:49:45.349ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
6322024-08-22T17:49:45.350ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
6332024-08-22T17:49:45.350ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
6342024-08-22T17:49:45.352ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6352024-08-22T17:49:45.355ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
6362024-08-22T17:49:45.355ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
6372024-08-22T17:49:45.355ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
6382024-08-22T17:49:45.358ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
6392024-08-22T17:49:45.358ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
6402024-08-22T17:49:45.360ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
6412024-08-22T17:49:45.360ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
6422024-08-22T17:49:45.361ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:454 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
6432024-08-22T17:49:45.361ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 295.746428ms iteration = 1 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
6442024-08-22T17:49:45.362ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
6452024-08-22T17:49:45.364ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
6462024-08-22T17:49:45.365ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
6472024-08-22T17:49:45.366ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
6482024-08-22T17:49:45.367ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
6492024-08-22T17:49:45.368ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
6502024-08-22T17:49:45.369ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
6512024-08-22T17:49:45.369ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
6522024-08-22T17:49:45.371ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
6532024-08-22T17:49:45.371ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
6542024-08-22T17:49:45.371ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6552024-08-22T17:49:45.372ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6562024-08-22T17:49:45.373ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
6572024-08-22T17:49:45.373ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
6582024-08-22T17:49:45.373ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
6592024-08-22T17:49:45.376ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
6602024-08-22T17:49:45.376ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
6612024-08-22T17:49:45.377ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
6622024-08-22T17:49:45.378ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
6632024-08-22T17:49:45.378ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
6642024-08-22T17:49:45.380ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
6652024-08-22T17:49:45.380ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
6662024-08-22T17:49:45.382ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6672024-08-22T17:49:45.385ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
6682024-08-22T17:49:45.385ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
6692024-08-22T17:49:45.385ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
6702024-08-22T17:49:45.389ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
6712024-08-22T17:49:45.389ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
6722024-08-22T17:49:45.392ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
6732024-08-22T17:49:45.392ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
6742024-08-22T17:49:45.394ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
6752024-08-22T17:49:45.395ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
6762024-08-22T17:49:45.397ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
6772024-08-22T17:49:45.398ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
6782024-08-22T17:49:45.405ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
6792024-08-22T17:49:45.406ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
6802024-08-22T17:49:45.407ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
6812024-08-22T17:49:45.407ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
6822024-08-22T17:49:45.409ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
6832024-08-22T17:49:45.410ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
6842024-08-22T17:49:45.411ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6852024-08-22T17:49:45.415ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
6862024-08-22T17:49:45.415ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
6872024-08-22T17:49:45.415ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
6882024-08-22T17:49:45.417ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
6892024-08-22T17:49:45.417ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
6902024-08-22T17:49:45.418ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
6912024-08-22T17:49:45.419ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
6922024-08-22T17:49:45.420ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6932024-08-22T17:49:45.423ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
6942024-08-22T17:49:45.423ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
6952024-08-22T17:49:45.423ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
6962024-08-22T17:49:45.454ZDEBGnexus (service-firewall-rule-progator): resolved names for firewall rules background_task = service_firewall_rule_propagation instance_interfaces = {} subnet_interfaces = {} subnet_networks = {Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })], Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })]} vpc_interfaces = {}
6972024-08-22T17:49:45.455ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
6982024-08-22T17:49:45.455ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
6992024-08-22T17:49:45.457ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7002024-08-22T17:49:45.458ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Fleet result = Ok(())
7012024-08-22T17:49:45.458ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
7022024-08-22T17:49:45.458ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
7032024-08-22T17:49:45.479ZDEBGnexus (service-firewall-rule-progator): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified. background_task = service_firewall_rule_propagation
7042024-08-22T17:49:45.479ZDEBGnexus (service-firewall-rule-progator): resolved firewall rules for sled agents background_task = service_firewall_rule_propagation sled_agent_rules = []
7052024-08-22T17:49:45.479ZDEBGnexus (service-firewall-rule-progator): resolved 0 rules for sleds background_task = service_firewall_rule_propagation
7062024-08-22T17:49:45.483ZDEBGnexus (BackgroundTasks): finished collection background_task = inventory_collection
7072024-08-22T17:49:45.483ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
7082024-08-22T17:49:45.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
7092024-08-22T17:49:45.485ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7102024-08-22T17:49:45.487ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
7112024-08-22T17:49:45.493ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
7122024-08-22T17:49:45.493ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
7132024-08-22T17:49:45.495ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7142024-08-22T17:49:45.498ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
7152024-08-22T17:49:45.498ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
7162024-08-22T17:49:45.499ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
7172024-08-22T17:49:45.500ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7182024-08-22T17:49:45.502ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
7192024-08-22T17:49:45.502ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
7202024-08-22T17:49:45.502ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
7212024-08-22T17:49:45.502ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
7222024-08-22T17:49:45.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
7232024-08-22T17:49:45.610ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 544.932656ms iteration = 1 status = {}
7242024-08-22T17:49:45.704ZDEBGnexus (service-firewall-rule-progator): resolved sleds for vpc oxide-services background_task = service_firewall_rule_propagation vpc_to_sled = []
7252024-08-22T17:49:45.704ZDEBGnexus (service-firewall-rule-progator): sending firewall rules to sled agents background_task = service_firewall_rule_propagation
7262024-08-22T17:49:45.704ZINFOnexus (service-firewall-rule-progator): updated firewall rules on 0 sleds background_task = service_firewall_rule_propagation file = nexus/networking/src/firewall_rules.rs:496
7272024-08-22T17:49:45.704ZDEBGnexus (service-firewall-rule-progator): successfully propagated service firewall rules background_task = service_firewall_rule_propagation elapsed = 632.188257ms
7282024-08-22T17:49:45.704ZDEBGnexus (BackgroundTasks): activation complete background_task = service_firewall_rule_propagation elapsed = 639.253764ms iteration = 1 status = {}
7292024-08-22T17:49:45.769ZINFOnexus (BackgroundTasks): inserted inventory collection background_task = inventory_collection collection_id = 704af88d-0cb7-4f6e-96e5-71ccba1bd64f file = nexus/db-queries/src/db/datastore/inventory.rs:893
7302024-08-22T17:49:45.770ZDEBGnexus (BackgroundTasks): inventory collection complete background_task = inventory_collection collection_id = 704af88d-0cb7-4f6e-96e5-71ccba1bd64f time_started = 2024-08-22 17:49:45.311836 UTC
7312024-08-22T17:49:45.770ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 703.817248ms iteration = 1 status = {"collection_id":"704af88d-0cb7-4f6e-96e5-71ccba1bd64f","time_done":"2024-08-22 17:49:45.483319 UTC","time_started":"2024-08-22 17:49:45.311836 UTC"}
7322024-08-22T17:49:45.770ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 2 reason = Dependency
7332024-08-22T17:49:45.770ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
7342024-08-22T17:49:45.770ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
7352024-08-22T17:49:45.771ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7362024-08-22T17:49:45.773ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
7372024-08-22T17:49:45.773ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
7382024-08-22T17:49:45.773ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
7392024-08-22T17:49:45.775ZINFOnexus (BackgroundTasks): Physical Disk Adoption: Rack not yet initialized background_task = physical_disk_adoption file = nexus/src/app/background/tasks/physical_disk_adoption.rs:79 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561
7402024-08-22T17:49:45.776ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 5.946301ms iteration = 2 status = {"error":"rack not yet initialized: c66d93a6-e2c3-4b19-9564-3b58bad50561"}
7412024-08-22T17:49:46.991ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
7422024-08-22T17:49:46.991ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
7432024-08-22T17:49:47.052ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
7442024-08-22T17:49:47.052ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
7452024-08-22T17:49:47.056ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7462024-08-22T17:49:47.059ZDEBGnexus (ServiceBalancer): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Rack { parent: Fleet, key: c66d93a6-e2c3-4b19-9564-3b58bad50561, lookup_type: ById(c66d93a6-e2c3-4b19-9564-3b58bad50561) } result = Ok(())
7472024-08-22T17:49:47.059ZINFOnexus: Still waiting for rack initialization: Rack { identity: RackIdentity { id: c66d93a6-e2c3-4b19-9564-3b58bad50561, time_created: 2024-08-22T17:49:45.371359Z, time_modified: 2024-08-22T17:49:45.371359Z }, initialized: false, tuf_base_url: None, rack_subnet: Some(V6(Ipv6Network { addr: fd00:1122:3344:100::, prefix: 56 })) } file = nexus/src/app/rack.rs:765
7482024-08-22T17:49:49.060ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
7492024-08-22T17:49:49.061ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
7502024-08-22T17:49:49.063ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
7512024-08-22T17:49:49.063ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
7522024-08-22T17:49:49.065ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7532024-08-22T17:49:49.067ZDEBGnexus (ServiceBalancer): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Rack { parent: Fleet, key: c66d93a6-e2c3-4b19-9564-3b58bad50561, lookup_type: ById(c66d93a6-e2c3-4b19-9564-3b58bad50561) } result = Ok(())
7542024-08-22T17:49:49.067ZINFOnexus: Still waiting for rack initialization: Rack { identity: RackIdentity { id: c66d93a6-e2c3-4b19-9564-3b58bad50561, time_created: 2024-08-22T17:49:45.371359Z, time_modified: 2024-08-22T17:49:45.371359Z }, initialized: false, tuf_base_url: None, rack_subnet: Some(V6(Ipv6Network { addr: fd00:1122:3344:100::, prefix: 56 })) } file = nexus/src/app/rack.rs:765
7552024-08-22T17:49:51.068ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
7562024-08-22T17:49:51.069ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
7572024-08-22T17:49:51.070ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
7582024-08-22T17:49:51.070ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
7592024-08-22T17:49:51.072ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7602024-08-22T17:49:51.074ZDEBGnexus (ServiceBalancer): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Rack { parent: Fleet, key: c66d93a6-e2c3-4b19-9564-3b58bad50561, lookup_type: ById(c66d93a6-e2c3-4b19-9564-3b58bad50561) } result = Ok(())
7612024-08-22T17:49:51.074ZINFOnexus: Still waiting for rack initialization: Rack { identity: RackIdentity { id: c66d93a6-e2c3-4b19-9564-3b58bad50561, time_created: 2024-08-22T17:49:45.371359Z, time_modified: 2024-08-22T17:49:45.371359Z }, initialized: false, tuf_base_url: None, rack_subnet: Some(V6(Ipv6Network { addr: fd00:1122:3344:100::, prefix: 56 })) } file = nexus/src/app/rack.rs:765
7622024-08-22T17:49:53.075ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
7632024-08-22T17:49:53.076ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
7642024-08-22T17:49:53.077ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
7652024-08-22T17:49:53.078ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
7662024-08-22T17:49:53.079ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7672024-08-22T17:49:53.081ZDEBGnexus (ServiceBalancer): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Rack { parent: Fleet, key: c66d93a6-e2c3-4b19-9564-3b58bad50561, lookup_type: ById(c66d93a6-e2c3-4b19-9564-3b58bad50561) } result = Ok(())
7682024-08-22T17:49:53.081ZINFOnexus: Still waiting for rack initialization: Rack { identity: RackIdentity { id: c66d93a6-e2c3-4b19-9564-3b58bad50561, time_created: 2024-08-22T17:49:45.371359Z, time_modified: 2024-08-22T17:49:45.371359Z }, initialized: false, tuf_base_url: None, rack_subnet: Some(V6(Ipv6Network { addr: fd00:1122:3344:100::, prefix: 56 })) } file = nexus/src/app/rack.rs:765
7692024-08-22T17:49:55.066ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 2 reason = Timeout
7702024-08-22T17:49:55.066ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 2 reason = Timeout
7712024-08-22T17:49:55.066ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
7722024-08-22T17:49:55.066ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
7732024-08-22T17:49:55.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
7742024-08-22T17:49:55.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
7752024-08-22T17:49:55.068ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7762024-08-22T17:49:55.071ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
7772024-08-22T17:49:55.071ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
7782024-08-22T17:49:55.071ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
7792024-08-22T17:49:55.073ZWARNnexus (BackgroundTasks): failed to read target blueprint background_task = blueprint_loader error = Internal Error: no target blueprint set file = nexus/src/app/background/tasks/blueprint_load.rs:69
7802024-08-22T17:49:55.073ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 7.340008ms iteration = 2 status = {"error":"failed to read target blueprint: Internal Error: no target blueprint set"}
7812024-08-22T17:49:55.082ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
7822024-08-22T17:49:55.082ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
7832024-08-22T17:49:55.084ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
7842024-08-22T17:49:55.084ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
7852024-08-22T17:49:55.085ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7862024-08-22T17:49:55.087ZDEBGnexus (ServiceBalancer): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Rack { parent: Fleet, key: c66d93a6-e2c3-4b19-9564-3b58bad50561, lookup_type: ById(c66d93a6-e2c3-4b19-9564-3b58bad50561) } result = Ok(())
7872024-08-22T17:49:55.087ZINFOnexus: Still waiting for rack initialization: Rack { identity: RackIdentity { id: c66d93a6-e2c3-4b19-9564-3b58bad50561, time_created: 2024-08-22T17:49:45.371359Z, time_modified: 2024-08-22T17:49:45.371359Z }, initialized: false, tuf_base_url: None, rack_subnet: Some(V6(Ipv6Network { addr: fd00:1122:3344:100::, prefix: 56 })) } file = nexus/src/app/rack.rs:765
7882024-08-22T17:49:55.093ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
7892024-08-22T17:49:55.093ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
7902024-08-22T17:49:55.123ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 57.323786ms iteration = 2 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
7912024-08-22T17:49:57.088ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
7922024-08-22T17:49:57.089ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
7932024-08-22T17:49:57.090ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
7942024-08-22T17:49:57.090ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
7952024-08-22T17:49:57.092ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7962024-08-22T17:49:57.094ZDEBGnexus (ServiceBalancer): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Rack { parent: Fleet, key: c66d93a6-e2c3-4b19-9564-3b58bad50561, lookup_type: ById(c66d93a6-e2c3-4b19-9564-3b58bad50561) } result = Ok(())
7972024-08-22T17:49:57.094ZINFOnexus: Still waiting for rack initialization: Rack { identity: RackIdentity { id: c66d93a6-e2c3-4b19-9564-3b58bad50561, time_created: 2024-08-22T17:49:45.371359Z, time_modified: 2024-08-22T17:49:45.371359Z }, initialized: false, tuf_base_url: None, rack_subnet: Some(V6(Ipv6Network { addr: fd00:1122:3344:100::, prefix: 56 })) } file = nexus/src/app/rack.rs:765
7982024-08-22T17:49:59.095ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
7992024-08-22T17:49:59.096ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
8002024-08-22T17:49:59.097ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
8012024-08-22T17:49:59.097ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
8022024-08-22T17:49:59.099ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8032024-08-22T17:49:59.101ZDEBGnexus (ServiceBalancer): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Rack { parent: Fleet, key: c66d93a6-e2c3-4b19-9564-3b58bad50561, lookup_type: ById(c66d93a6-e2c3-4b19-9564-3b58bad50561) } result = Ok(())
8042024-08-22T17:49:59.101ZINFOnexus: Rack initialized file = nexus/src/app/rack.rs:762
8052024-08-22T17:49:59.101ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
8062024-08-22T17:49:59.101ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
8072024-08-22T17:49:59.103ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
8082024-08-22T17:49:59.104ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
8092024-08-22T17:49:59.106ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
8102024-08-22T17:49:59.106ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
8112024-08-22T17:49:59.107ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
8122024-08-22T17:49:59.108ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
8132024-08-22T17:49:59.109ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
8142024-08-22T17:49:59.109ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
8152024-08-22T17:49:59.110ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
8162024-08-22T17:49:59.110ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
8172024-08-22T17:49:59.111ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8182024-08-22T17:49:59.114ZDEBGnexus (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
8192024-08-22T17:49:59.114ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
8202024-08-22T17:49:59.114ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
8212024-08-22T17:49:59.115ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
8222024-08-22T17:49:59.115ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
8232024-08-22T17:49:59.116ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
8242024-08-22T17:49:59.116ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
8252024-08-22T17:49:59.118ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8262024-08-22T17:49:59.120ZDEBGnexus (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
8272024-08-22T17:49:59.120ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
8282024-08-22T17:49:59.120ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
8292024-08-22T17:49:59.122ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
8302024-08-22T17:49:59.122ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
8312024-08-22T17:49:59.123ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
8322024-08-22T17:49:59.123ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
8332024-08-22T17:49:59.124ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
8342024-08-22T17:49:59.124ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
8352024-08-22T17:49:59.126ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
8362024-08-22T17:49:59.126ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
8372024-08-22T17:49:59.127ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
8382024-08-22T17:49:59.127ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
8392024-08-22T17:49:59.128ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
8402024-08-22T17:49:59.128ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
8412024-08-22T17:49:59.129ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8422024-08-22T17:49:59.132ZDEBGnexus (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
8432024-08-22T17:49:59.132ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
8442024-08-22T17:49:59.132ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
8452024-08-22T17:49:59.133ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
8462024-08-22T17:49:59.133ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
8472024-08-22T17:49:59.134ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
8482024-08-22T17:49:59.135ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
8492024-08-22T17:49:59.137ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
8502024-08-22T17:49:59.137ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
8512024-08-22T17:49:59.139ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
8522024-08-22T17:49:59.139ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
8532024-08-22T17:49:59.140ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
8542024-08-22T17:49:59.140ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
8552024-08-22T17:49:59.141ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
8562024-08-22T17:49:59.141ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
8572024-08-22T17:49:59.143ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8582024-08-22T17:49:59.146ZDEBGnexus (InternalApi): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
8592024-08-22T17:49:59.146ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
8602024-08-22T17:49:59.146ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
8612024-08-22T17:49:59.147ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
8622024-08-22T17:49:59.147ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
8632024-08-22T17:49:59.148ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
8642024-08-22T17:49:59.148ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
8652024-08-22T17:49:59.150ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8662024-08-22T17:49:59.152ZDEBGnexus (InternalApi): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
8672024-08-22T17:49:59.152ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
8682024-08-22T17:49:59.153ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
8692024-08-22T17:49:59.160ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
8702024-08-22T17:49:59.160ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
8712024-08-22T17:49:59.161ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
8722024-08-22T17:49:59.162ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
8732024-08-22T17:49:59.163ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
8742024-08-22T17:49:59.163ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
8752024-08-22T17:49:59.164ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
8762024-08-22T17:49:59.164ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
8772024-08-22T17:49:59.166ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
8782024-08-22T17:49:59.166ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
8792024-08-22T17:49:59.168ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
8802024-08-22T17:49:59.168ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
8812024-08-22T17:49:59.169ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
8822024-08-22T17:49:59.169ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
8832024-08-22T17:49:59.171ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8842024-08-22T17:49:59.174ZDEBGnexus (InternalApi): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
8852024-08-22T17:49:59.174ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
8862024-08-22T17:49:59.174ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
8872024-08-22T17:49:59.175ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
8882024-08-22T17:49:59.175ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
8892024-08-22T17:49:59.176ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
8902024-08-22T17:49:59.176ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
8912024-08-22T17:49:59.177ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8922024-08-22T17:49:59.180ZDEBGnexus (InternalApi): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
8932024-08-22T17:49:59.180ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
8942024-08-22T17:49:59.180ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
8952024-08-22T17:49:59.193ZDEBGnexus (InternalApi): resolved names for firewall rules instance_interfaces = {} subnet_interfaces = {Name("nexus"): [NetworkInterface { id: 09f89158-2ea8-4bfd-911c-3bf3ffef2aa4, kind: Service { id: 1fa445cf-528e-4b77-85ed-445f0a227ac1 }, name: Name("nexus-1fa445cf-528e-4b77-85ed-445f0a227ac1"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 205, 230])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 5c255d57-c119-4de4-b8a2-7112b0b74875, kind: Service { id: eaa260db-2b19-4ae1-97e8-5735c9f5d955 }, name: Name("nexus-eaa260db-2b19-4ae1-97e8-5735c9f5d955"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 249, 182])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 5d88cb36-556b-42c6-a63f-36eaaaebfa41, kind: Service { id: 21688317-2226-405d-92aa-f8b1720a4f0c }, name: Name("nexus-21688317-2226-405d-92aa-f8b1720a4f0c"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 138, 172])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], Name("external-dns"): [NetworkInterface { id: 88350ddb-4ac1-44fa-af24-3dfceef6dec3, kind: Service { id: afac2eab-cadf-48d0-94a7-ad562ade88c6 }, name: Name("external-dns-afac2eab-cadf-48d0-94a7-ad562ade88c6"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 169, 222])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: bc0e4e8b-293a-4626-9868-98cd4702f955, kind: Service { id: 9488ce0a-6732-4f75-baf9-4ecfa406df91 }, name: Name("external-dns-9488ce0a-6732-4f75-baf9-4ecfa406df91"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 223, 41])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }]} subnet_networks = {Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })], Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })]} vpc_interfaces = {}
8962024-08-22T17:49:59.193ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
8972024-08-22T17:49:59.193ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
8982024-08-22T17:49:59.194ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8992024-08-22T17:49:59.196ZDEBGnexus (InternalApi): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet result = Ok(())
9002024-08-22T17:49:59.196ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
9012024-08-22T17:49:59.196ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
9022024-08-22T17:49:59.205ZDEBGnexus (InternalApi): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified.
9032024-08-22T17:49:59.208ZDEBGnexus (InternalApi): resolved firewall rules for sled agents sled_agent_rules = [ResolvedVpcFirewallRule { action: Allow, direction: Inbound, filter_hosts: None, filter_ports: Some([L4PortRange("53")]), filter_protocols: Some([Udp]), priority: 65534, status: Enabled, targets: [NetworkInterface { id: 88350ddb-4ac1-44fa-af24-3dfceef6dec3, kind: Service { id: afac2eab-cadf-48d0-94a7-ad562ade88c6 }, name: Name("external-dns-afac2eab-cadf-48d0-94a7-ad562ade88c6"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 169, 222])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: bc0e4e8b-293a-4626-9868-98cd4702f955, kind: Service { id: 9488ce0a-6732-4f75-baf9-4ecfa406df91 }, name: Name("external-dns-9488ce0a-6732-4f75-baf9-4ecfa406df91"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 223, 41])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }] }, ResolvedVpcFirewallRule { action: Allow, direction: Inbound, filter_hosts: None, filter_ports: Some([L4PortRange("80"), L4PortRange("443")]), filter_protocols: Some([Tcp]), priority: 65534, status: Enabled, targets: [NetworkInterface { id: 09f89158-2ea8-4bfd-911c-3bf3ffef2aa4, kind: Service { id: 1fa445cf-528e-4b77-85ed-445f0a227ac1 }, name: Name("nexus-1fa445cf-528e-4b77-85ed-445f0a227ac1"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 205, 230])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 5c255d57-c119-4de4-b8a2-7112b0b74875, kind: Service { id: eaa260db-2b19-4ae1-97e8-5735c9f5d955 }, name: Name("nexus-eaa260db-2b19-4ae1-97e8-5735c9f5d955"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 249, 182])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 5d88cb36-556b-42c6-a63f-36eaaaebfa41, kind: Service { id: 21688317-2226-405d-92aa-f8b1720a4f0c }, name: Name("nexus-21688317-2226-405d-92aa-f8b1720a4f0c"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 138, 172])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }] }]
9042024-08-22T17:49:59.208ZDEBGnexus (InternalApi): resolved 2 rules for sleds
9052024-08-22T17:49:59.285ZDEBGnexus (InternalApi): resolved sleds for vpc oxide-services vpc_to_sled = [Sled { identity: SledIdentity { id: c961ede4-7532-4ca1-95a5-690b4d7c9b5a, time_created: 2024-08-22T17:49:54.055103Z, time_modified: 2024-08-22T17:49:54.055103Z }, time_deleted: None, rcgen: Generation(Generation(19)), rack_id: c66d93a6-e2c3-4b19-9564-3b58bad50561, is_scrimlet: true, serial_number: "buskin", part_number: "i86pc", revision: SqlU32(0), usable_hardware_threads: SqlU32(48), usable_physical_ram: ByteCount(ByteCount(102952026112)), reservoir_size: ByteCount(ByteCount(51474595840)), ip: fd00:1122:3344:101::1, port: SqlU16(12345), last_used_address: fd00:1122:3344:101::ffff, policy: InService, state: Active, sled_agent_gen: Generation(Generation(1)) }]
9062024-08-22T17:49:59.285ZDEBGnexus (InternalApi): sending firewall rules to sled agents
9072024-08-22T17:49:59.285ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
9082024-08-22T17:49:59.285ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
9092024-08-22T17:49:59.287ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
9102024-08-22T17:49:59.287ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
9112024-08-22T17:49:59.289ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9122024-08-22T17:49:59.292ZDEBGnexus (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Sled { parent: Fleet, key: c961ede4-7532-4ca1-95a5-690b4d7c9b5a, lookup_type: ById(c961ede4-7532-4ca1-95a5-690b4d7c9b5a) } result = Ok(())
9132024-08-22T17:49:59.327ZDEBGnexus (InternalApi): client request SledAgent = c961ede4-7532-4ca1-95a5-690b4d7c9b5a body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules
9142024-08-22T17:49:59.328ZDEBGnexus (InternalApi): client response SledAgent = c961ede4-7532-4ca1-95a5-690b4d7c9b5a result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/vpc/001de000-074c-4000-8000-000000000000/firewall/rules", query: None, fragment: None }, status: 204, headers: {"x-request-id": "3918c9b2-42a0-4b3e-81de-cea0e1e957de", "date": "Thu, 22 Aug 2024 17:49:58 GMT"} })
9152024-08-22T17:49:59.328ZINFOnexus (InternalApi): updated firewall rules on 1 sleds file = nexus/networking/src/firewall_rules.rs:496
9162024-08-22T17:49:59.329ZINFOnexus: plumbed initial IP allowlist file = nexus/src/app/allow_list.rs:161
9172024-08-22T17:50:05.066ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 3 reason = Timeout
9182024-08-22T17:50:05.066ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 3 reason = Timeout
9192024-08-22T17:50:05.066ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
9202024-08-22T17:50:05.066ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9212024-08-22T17:50:05.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
9222024-08-22T17:50:05.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9232024-08-22T17:50:05.068ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9242024-08-22T17:50:05.071ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
9252024-08-22T17:50:05.071ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9262024-08-22T17:50:05.071ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9272024-08-22T17:50:05.076ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9282024-08-22T17:50:05.077ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9292024-08-22T17:50:05.079ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9302024-08-22T17:50:05.081ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 147bdc10-344c-4ee8-a219-603cfc66e6d2, lookup_type: ById(147bdc10-344c-4ee8-a219-603cfc66e6d2) } result = Ok(())
9312024-08-22T17:50:05.081ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9322024-08-22T17:50:05.081ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9332024-08-22T17:50:05.102ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
9342024-08-22T17:50:05.102ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
9352024-08-22T17:50:05.128ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 62.239551ms iteration = 3 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
9362024-08-22T17:50:05.135ZINFOnexus (BackgroundTasks): found new target blueprint (first find) background_task = blueprint_loader file = nexus/src/app/background/tasks/blueprint_load.rs:87 target_id = 147bdc10-344c-4ee8-a219-603cfc66e6d2 time_created = 2024-08-22 17:49:18.445237 UTC
9372024-08-22T17:50:05.135ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 2 reason = Dependency
9382024-08-22T17:50:05.135ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:80 reason = blueprint disabled target_id = 147bdc10-344c-4ee8-a219-603cfc66e6d2
9392024-08-22T17:50:05.135ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 11.191\u{b5}s iteration = 2 status = {"error":"blueprint disabled","target_id":"147bdc10-344c-4ee8-a219-603cfc66e6d2"}
9402024-08-22T17:50:05.135ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 69.241468ms iteration = 3 status = {"status":"first target blueprint","target_id":"147bdc10-344c-4ee8-a219-603cfc66e6d2","time_created":"2024-08-22T17:49:18.445237Z","time_found":"2024-08-22T17:50:05.135585357Z"}
9412024-08-22T17:50:05.135ZDEBGnexus (BackgroundTasks): activating background_task = crdb_node_id_collector iteration = 2 reason = Dependency
9422024-08-22T17:50:05.137ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
9432024-08-22T17:50:05.137ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
9442024-08-22T17:50:05.137ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
9452024-08-22T17:50:05.138ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
9462024-08-22T17:50:05.138ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
9472024-08-22T17:50:05.138ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
9482024-08-22T17:50:05.138ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
9492024-08-22T17:50:05.138ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
9502024-08-22T17:50:05.138ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
9512024-08-22T17:50:05.138ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
9522024-08-22T17:50:05.139ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9532024-08-22T17:50:05.141ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
9542024-08-22T17:50:05.141ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
9552024-08-22T17:50:05.141ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
9562024-08-22T17:50:05.141ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9572024-08-22T17:50:05.143ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
9582024-08-22T17:50:05.143ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
9592024-08-22T17:50:05.143ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
9602024-08-22T17:50:05.143ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9612024-08-22T17:50:05.144ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
9622024-08-22T17:50:05.144ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
9632024-08-22T17:50:05.144ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
9642024-08-22T17:50:05.144ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9652024-08-22T17:50:05.146ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
9662024-08-22T17:50:05.146ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
9672024-08-22T17:50:05.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
9682024-08-22T17:50:05.146ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9692024-08-22T17:50:05.147ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
9702024-08-22T17:50:05.147ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
9712024-08-22T17:50:05.147ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
9722024-08-22T17:50:05.180ZDEBGnexus (BackgroundTasks): client request background_task = crdb_node_id_collector body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/node/id
9732024-08-22T17:50:05.212ZDEBGnexus (BackgroundTasks): client request background_task = crdb_node_id_collector body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/node/id
9742024-08-22T17:50:05.244ZDEBGnexus (BackgroundTasks): client request background_task = crdb_node_id_collector body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/node/id
9752024-08-22T17:50:05.280ZDEBGnexus (BackgroundTasks): client request background_task = crdb_node_id_collector body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/node/id
9762024-08-22T17:50:05.313ZDEBGnexus (BackgroundTasks): client request background_task = crdb_node_id_collector body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/node/id
9772024-08-22T17:50:06.309ZDEBGnexus (BackgroundTasks): client response background_task = crdb_node_id_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::6)), port: Some(32222), path: "/node/id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "53d65982-0f88-41ea-ac50-ebde3cff0f81", "content-length": "64", "date": "Thu, 22 Aug 2024 17:50:05 GMT"} })
9782024-08-22T17:50:06.309ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
9792024-08-22T17:50:06.310ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
9802024-08-22T17:50:06.311ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9812024-08-22T17:50:06.313ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
9822024-08-22T17:50:06.313ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
9832024-08-22T17:50:06.313ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
9842024-08-22T17:50:06.329ZDEBGnexus (BackgroundTasks): client response background_task = crdb_node_id_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::3)), port: Some(32222), path: "/node/id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "2ef194c9-ff14-4601-b4f2-d03d915ec3e7", "content-length": "64", "date": "Thu, 22 Aug 2024 17:50:06 GMT"} })
9852024-08-22T17:50:06.329ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
9862024-08-22T17:50:06.329ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
9872024-08-22T17:50:06.331ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9882024-08-22T17:50:06.332ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
9892024-08-22T17:50:06.332ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
9902024-08-22T17:50:06.332ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
9912024-08-22T17:50:06.391ZDEBGnexus (BackgroundTasks): client response background_task = crdb_node_id_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::7)), port: Some(32222), path: "/node/id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "c271ce6e-4cb4-45ea-9bfd-77cc65fcc70a", "content-length": "64", "date": "Thu, 22 Aug 2024 17:50:06 GMT"} })
9922024-08-22T17:50:06.391ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
9932024-08-22T17:50:06.392ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
9942024-08-22T17:50:06.393ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9952024-08-22T17:50:06.394ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
9962024-08-22T17:50:06.394ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
9972024-08-22T17:50:06.394ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
9982024-08-22T17:50:06.429ZDEBGnexus (BackgroundTasks): client response background_task = crdb_node_id_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::5)), port: Some(32222), path: "/node/id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "a512d301-9646-4759-ad47-d1b31b13dbc7", "content-length": "64", "date": "Thu, 22 Aug 2024 17:50:06 GMT"} })
9992024-08-22T17:50:06.429ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
10002024-08-22T17:50:06.429ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
10012024-08-22T17:50:06.431ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10022024-08-22T17:50:06.432ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
10032024-08-22T17:50:06.432ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
10042024-08-22T17:50:06.432ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
10052024-08-22T17:50:15.066ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 4 reason = Timeout
10062024-08-22T17:50:15.066ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
10072024-08-22T17:50:15.066ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 2 reason = Timeout
10082024-08-22T17:50:15.066ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 2 reason = Timeout
10092024-08-22T17:50:15.066ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
10102024-08-22T17:50:15.066ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 2 reason = Timeout
10112024-08-22T17:50:15.066ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
10122024-08-22T17:50:15.066ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
10132024-08-22T17:50:15.066ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 2 reason = Timeout
10142024-08-22T17:50:15.066ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 2 reason = Timeout
10152024-08-22T17:50:15.066ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
10162024-08-22T17:50:15.066ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 3 reason = Timeout
10172024-08-22T17:50:15.066ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 2 reason = Timeout
10182024-08-22T17:50:15.066ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
10192024-08-22T17:50:15.066ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 4 reason = Timeout
10202024-08-22T17:50:15.066ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
10212024-08-22T17:50:15.066ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
10222024-08-22T17:50:15.066ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 2 reason = Timeout
10232024-08-22T17:50:15.066ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 2 reason = Timeout
10242024-08-22T17:50:15.066ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 2 reason = Timeout
10252024-08-22T17:50:15.066ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
10262024-08-22T17:50:15.066ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
10272024-08-22T17:50:15.066ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 2 reason = Timeout
10282024-08-22T17:50:15.066ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
10292024-08-22T17:50:15.066ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 2 reason = Timeout
10302024-08-22T17:50:15.066ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
10312024-08-22T17:50:15.066ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
10322024-08-22T17:50:15.066ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
10332024-08-22T17:50:15.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
10342024-08-22T17:50:15.067ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
10352024-08-22T17:50:15.067ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
10362024-08-22T17:50:15.067ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
10372024-08-22T17:50:15.067ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
10382024-08-22T17:50:15.067ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
10392024-08-22T17:50:15.067ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
10402024-08-22T17:50:15.067ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
10412024-08-22T17:50:15.067ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
10422024-08-22T17:50:15.067ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
10432024-08-22T17:50:15.067ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
10442024-08-22T17:50:15.067ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
10452024-08-22T17:50:15.067ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
10462024-08-22T17:50:15.068ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10472024-08-22T17:50:15.068ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 2 reason = Timeout
10482024-08-22T17:50:15.068ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
10492024-08-22T17:50:15.069ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
10502024-08-22T17:50:15.070ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10512024-08-22T17:50:15.070ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 4.023451ms iteration = 2 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
10522024-08-22T17:50:15.071ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
10532024-08-22T17:50:15.071ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
10542024-08-22T17:50:15.071ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
10552024-08-22T17:50:15.073ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
10562024-08-22T17:50:15.073ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
10572024-08-22T17:50:15.073ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
10582024-08-22T17:50:15.073ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10592024-08-22T17:50:15.074ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10602024-08-22T17:50:15.075ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-c961ede4-7532-4ca1-95a5-690b4d7c9b5a.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-c961ede4-7532-4ca1-95a5-690b4d7c9b5a.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1603, tv_nsec: 610294249 } })
10612024-08-22T17:50:15.076ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1050
10622024-08-22T17:50:15.077ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
10632024-08-22T17:50:15.077ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
10642024-08-22T17:50:15.077ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
10652024-08-22T17:50:15.077ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
10662024-08-22T17:50:15.077ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
10672024-08-22T17:50:15.078ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
10682024-08-22T17:50:15.079ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10692024-08-22T17:50:15.079ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10702024-08-22T17:50:15.081ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
10712024-08-22T17:50:15.081ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
10722024-08-22T17:50:15.082ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
10732024-08-22T17:50:15.082ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
10742024-08-22T17:50:15.082ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
10752024-08-22T17:50:15.083ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-c961ede4-7532-4ca1-95a5-690b4d7c9b5a.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-c961ede4-7532-4ca1-95a5-690b4d7c9b5a.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1603, tv_nsec: 617891028 } })
10762024-08-22T17:50:15.083ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
10772024-08-22T17:50:15.083ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
10782024-08-22T17:50:15.083ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
10792024-08-22T17:50:15.084ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1050 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561
10802024-08-22T17:50:15.085ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
10812024-08-22T17:50:15.085ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
10822024-08-22T17:50:15.085ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10832024-08-22T17:50:15.086ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-c961ede4-7532-4ca1-95a5-690b4d7c9b5a.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-c961ede4-7532-4ca1-95a5-690b4d7c9b5a.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1603, tv_nsec: 621011894 } })
10842024-08-22T17:50:15.088ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1050
10852024-08-22T17:50:15.089ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 147bdc10-344c-4ee8-a219-603cfc66e6d2, lookup_type: ById(147bdc10-344c-4ee8-a219-603cfc66e6d2) } result = Ok(())
10862024-08-22T17:50:15.089ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
10872024-08-22T17:50:15.089ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
10882024-08-22T17:50:15.099ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
10892024-08-22T17:50:15.099ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
10902024-08-22T17:50:15.105ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10912024-08-22T17:50:15.108ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
10922024-08-22T17:50:15.108ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
10932024-08-22T17:50:15.108ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
10942024-08-22T17:50:15.110ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
10952024-08-22T17:50:15.110ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
10962024-08-22T17:50:15.114ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
10972024-08-22T17:50:15.114ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
10982024-08-22T17:50:15.115ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 49.337375ms iteration = 2 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
10992024-08-22T17:50:15.118ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:55
11002024-08-22T17:50:15.118ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
11012024-08-22T17:50:15.118ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
11022024-08-22T17:50:15.119ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
11032024-08-22T17:50:15.127ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 58.462972ms iteration = 2 status = {"errors":[],"garbage_collect_requested":[]}
11042024-08-22T17:50:15.128ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
11052024-08-22T17:50:15.128ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
11062024-08-22T17:50:15.137ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
11072024-08-22T17:50:15.137ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
11082024-08-22T17:50:15.139ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11092024-08-22T17:50:15.141ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
11102024-08-22T17:50:15.141ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
11112024-08-22T17:50:15.141ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
11122024-08-22T17:50:15.144ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
11132024-08-22T17:50:15.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
11142024-08-22T17:50:15.146ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11152024-08-22T17:50:15.149ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1058 zone_address = fd00:1122:3344:101::2
11162024-08-22T17:50:15.149ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
11172024-08-22T17:50:15.149ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "09c1389a-214a-46f5-b208-9a44746c5a3b", "content-length": "26", "date": "Thu, 22 Aug 2024 17:50:14 GMT"} })
11182024-08-22T17:50:15.149ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1063 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
11192024-08-22T17:50:15.150ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: c961ede4-7532-4ca1-95a5-690b4d7c9b5a, lookup_type: ById(c961ede4-7532-4ca1-95a5-690b4d7c9b5a) } result = Ok(())
11202024-08-22T17:50:15.151ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1095 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
11212024-08-22T17:50:15.152ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
11222024-08-22T17:50:15.152ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:55
11232024-08-22T17:50:15.152ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:220 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
11242024-08-22T17:50:15.152ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 85.776693ms iteration = 2 status = {"destroyed_active_vmms":0,"error":null,"saga_start_failures":0,"sagas_completed":0,"sagas_failed":0,"sagas_started":0,"terminated_active_migrations":0}
11252024-08-22T17:50:15.152ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
11262024-08-22T17:50:15.155ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 89.347671ms iteration = 4 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
11272024-08-22T17:50:15.161ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
11282024-08-22T17:50:15.161ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
11292024-08-22T17:50:15.166ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 100.062817ms iteration = 2 status = {"region_replacement_started_err":0,"region_replacement_started_ok":0}
11302024-08-22T17:50:15.178ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:454 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
11312024-08-22T17:50:15.178ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 111.936939ms iteration = 2 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
11322024-08-22T17:50:15.182ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
11332024-08-22T17:50:15.183ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
11342024-08-22T17:50:15.183ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1058 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 zone_address = fd00:1122:3344:101::2
11352024-08-22T17:50:15.183ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
11362024-08-22T17:50:15.183ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "d8efe89a-fab9-4549-aaeb-17fd64e61492", "content-length": "26", "date": "Thu, 22 Aug 2024 17:50:15 GMT"} })
11372024-08-22T17:50:15.183ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1063 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
11382024-08-22T17:50:15.184ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1095 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561
11392024-08-22T17:50:15.192ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
11402024-08-22T17:50:15.192ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
11412024-08-22T17:50:15.206ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
11422024-08-22T17:50:15.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
11432024-08-22T17:50:15.207ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
11442024-08-22T17:50:15.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
11452024-08-22T17:50:15.208ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 147bdc10-344c-4ee8-a219-603cfc66e6d2 original_time_created = 2024-08-22 17:49:18.445237 UTC target_id = 147bdc10-344c-4ee8-a219-603cfc66e6d2 time_created = 2024-08-22 17:49:18.445237 UTC
11462024-08-22T17:50:15.208ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 141.769913ms iteration = 4 status = {"status":"target blueprint unchanged","target_id":"147bdc10-344c-4ee8-a219-603cfc66e6d2","time_created":"2024-08-22T17:49:18.445237Z"}
11472024-08-22T17:50:15.216ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
11482024-08-22T17:50:15.216ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
11492024-08-22T17:50:15.222ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
11502024-08-22T17:50:15.222ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
11512024-08-22T17:50:15.226ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 159.981187ms iteration = 3 status = {"physical_disks_added":0}
11522024-08-22T17:50:15.227ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1058 zone_address = fd00:1122:3344:101::2
11532024-08-22T17:50:15.227ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
11542024-08-22T17:50:15.228ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "7be55abc-4668-4754-9518-8910ccbeca1e", "content-length": "26", "date": "Thu, 22 Aug 2024 17:50:15 GMT"} })
11552024-08-22T17:50:15.228ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1063 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
11562024-08-22T17:50:15.230ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1095 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
11572024-08-22T17:50:15.247ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
11582024-08-22T17:50:15.247ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
11592024-08-22T17:50:15.289ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 223.242783ms iteration = 2 status = {}
11602024-08-22T17:50:15.308ZDEBGnexus (BackgroundTasks): client request SledAgent = c961ede4-7532-4ca1-95a5-690b4d7c9b5a background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
11612024-08-22T17:50:15.308ZDEBGnexus (BackgroundTasks): client response SledAgent = c961ede4-7532-4ca1-95a5-690b4d7c9b5a background_task = vpc_route_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/vpc-routes", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "da310593-9e25-40b6-9c75-0b768dcd12cc", "content-length": "372", "date": "Thu, 22 Aug 2024 17:50:14 GMT"} })
11622024-08-22T17:50:15.310ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11632024-08-22T17:50:15.312ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
11642024-08-22T17:50:15.312ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
11652024-08-22T17:50:15.312ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
11662024-08-22T17:50:15.313ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
11672024-08-22T17:50:15.313ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
11682024-08-22T17:50:15.314ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11692024-08-22T17:50:15.315ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
11702024-08-22T17:50:15.315ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
11712024-08-22T17:50:15.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
11722024-08-22T17:50:15.316ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
11732024-08-22T17:50:15.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
11742024-08-22T17:50:15.317ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-c961ede4-7532-4ca1-95a5-690b4d7c9b5a.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-c961ede4-7532-4ca1-95a5-690b4d7c9b5a.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1603, tv_nsec: 851770796 } })
11752024-08-22T17:50:15.318ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = service_zone_nat_tracker file = nexus/src/app/mod.rs:1050
11762024-08-22T17:50:15.319ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
11772024-08-22T17:50:15.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
11782024-08-22T17:50:15.321ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
11792024-08-22T17:50:15.322ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "8d2ad5c5-b8c1-4b85-a80a-a585a8699e21", "content-length": "1", "date": "Thu, 22 Aug 2024 17:50:15 GMT"} })
11802024-08-22T17:50:15.323ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
11812024-08-22T17:50:15.323ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
11822024-08-22T17:50:15.324ZDEBGnexus (BackgroundTasks): client request SledAgent = c961ede4-7532-4ca1-95a5-690b4d7c9b5a background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
11832024-08-22T17:50:15.324ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 258.289144ms iteration = 2 status = null
11842024-08-22T17:50:15.325ZDEBGnexus (BackgroundTasks): client response SledAgent = c961ede4-7532-4ca1-95a5-690b4d7c9b5a background_task = vpc_route_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/vpc-routes", query: None, fragment: None }, status: 204, headers: {"x-request-id": "f74388cb-ba27-4d8c-b8ca-0087ec6b1dc6", "date": "Thu, 22 Aug 2024 17:50:14 GMT"} })
11852024-08-22T17:50:15.326ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 259.832262ms iteration = 2 status = {}
11862024-08-22T17:50:15.415ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = service_zone_nat_tracker file = nexus/src/app/mod.rs:1058 zone_address = fd00:1122:3344:101::2
11872024-08-22T17:50:15.415ZDEBGnexus (MgsClient): client request background_task = service_zone_nat_tracker body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
11882024-08-22T17:50:15.416ZDEBGnexus (MgsClient): client response background_task = service_zone_nat_tracker result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "afb6ae3d-274d-493b-a47d-36eb5dee1b29", "content-length": "26", "date": "Thu, 22 Aug 2024 17:50:15 GMT"} })
11892024-08-22T17:50:15.416ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = service_zone_nat_tracker file = nexus/src/app/mod.rs:1063 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
11902024-08-22T17:50:15.417ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = service_zone_nat_tracker file = nexus/src/app/mod.rs:1095 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
11912024-08-22T17:50:15.498ZDEBGnexus (DpdClient): client request background_task = service_zone_nat_tracker body = None method = POST uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/trigger
11922024-08-22T17:50:15.500ZDEBGnexus (DpdClient): client response background_task = service_zone_nat_tracker result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/trigger", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "9b037ba4-1938-4275-a367-228d02aec326", "content-length": "4", "date": "Thu, 22 Aug 2024 17:50:15 GMT"} })
11932024-08-22T17:50:15.504ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 437.693621ms iteration = 2 status = 6
11942024-08-22T17:50:15.507ZDEBGnexus (BackgroundTasks): client request SledAgent = c961ede4-7532-4ca1-95a5-690b4d7c9b5a background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
11952024-08-22T17:50:15.536ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
11962024-08-22T17:50:15.536ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
11972024-08-22T17:50:15.539ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:147 port = SwitchPort { id: c772d372-7448-4ce0-b8d1-efa645d18ed1, rack_id: c66d93a6-e2c3-4b19-9564-3b58bad50561, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(9ff05247-84aa-47d8-8619-ce0a8dbbaf7c) } rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 switch_location = Switch0
11982024-08-22T17:50:15.539ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
11992024-08-22T17:50:15.539ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
12002024-08-22T17:50:15.556ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
12012024-08-22T17:50:15.557ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/port/qsfp0/settings", query: Some("tag=omicron"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "3eac07bd-b8cd-4bf4-8176-da9bd5fae009", "content-length": "121", "date": "Thu, 22 Aug 2024 17:50:15 GMT"} })
12022024-08-22T17:50:15.557ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1714 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G } }} } switch_port_id = "qsfp0"
12032024-08-22T17:50:15.557ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1741 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 9ff05247-84aa-47d8-8619-ce0a8dbbaf7c
12042024-08-22T17:50:15.559ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:379 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
12052024-08-22T17:50:15.559ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
12062024-08-22T17:50:15.559ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:392 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 routes = {}
12072024-08-22T17:50:15.559ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:398 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 routes = {}
12082024-08-22T17:50:15.559ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:418 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561
12092024-08-22T17:50:15.559ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
12102024-08-22T17:50:15.560ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
12112024-08-22T17:50:15.564ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
12122024-08-22T17:50:15.565ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv4", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "333e6b45-f0a0-411b-95aa-84c9cd194cc2", "content-length": "2", "date": "Thu, 22 Aug 2024 17:50:15 GMT"} })
12132024-08-22T17:50:15.565ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
12142024-08-22T17:50:15.565ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "e02ad2c4-256c-40be-8463-481b887c6fed", "content-length": "52", "date": "Thu, 22 Aug 2024 17:50:15 GMT"} })
12152024-08-22T17:50:15.566ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0" }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:467 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 switch_location = Switch0
12162024-08-22T17:50:15.566ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
12172024-08-22T17:50:15.631ZDEBGnexus (BackgroundTasks): client response SledAgent = c961ede4-7532-4ca1-95a5-690b4d7c9b5a background_task = v2p_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/v2p", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "4abd7f88-026a-4866-8913-416f7344270d", "content-length": "691", "date": "Thu, 22 Aug 2024 17:50:15 GMT"} })
12182024-08-22T17:50:15.631ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = buskin
12192024-08-22T17:50:15.631ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = buskin
12202024-08-22T17:50:15.631ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = buskin
12212024-08-22T17:50:15.632ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 566.032654ms iteration = 2 status = {}
12222024-08-22T17:50:15.735ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/switch-ports", query: None, fragment: None }, status: 204, headers: {"x-request-id": "7e50b7c3-1ed6-4195-92eb-6528390b33f6", "date": "Thu, 22 Aug 2024 17:50:15 GMT"} })
12232024-08-22T17:50:15.735ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:770 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 switch = Switch0
12242024-08-22T17:50:15.735ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
12252024-08-22T17:50:15.735ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/network-bootstore-config", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "c3dc610c-b591-4754-a661-eaaf7b5a2480", "content-length": "524", "date": "Thu, 22 Aug 2024 17:50:15 GMT"} })
12262024-08-22T17:50:15.736ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
12272024-08-22T17:50:15.736ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
12282024-08-22T17:50:15.739ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
12292024-08-22T17:50:15.740ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
12302024-08-22T17:50:15.743ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
12312024-08-22T17:50:15.743ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
12322024-08-22T17:50:15.745ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
12332024-08-22T17:50:15.745ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
12342024-08-22T17:50:15.766ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 699.367293ms iteration = 2 status = {}
12352024-08-22T17:50:20.315ZDEBGnexus (BackgroundTasks): client response background_task = crdb_node_id_collector result = Err(reqwest::Error { kind: Request, url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::4)), port: Some(32222), path: "/node/id", query: None, fragment: None }, source: TimedOut })
12362024-08-22T17:50:20.318ZDEBGnexus (BackgroundTasks): activation complete background_task = crdb_node_id_collector elapsed = 15.182568877s iteration = 2 status = {"errors":[{"err":"failed to fetch node ID for zone aa7c602b-6b2c-4ab0-8f78-6e74bc723578 at http://[fd00:1122:3344:101::4]:32222: Communication Error: error sending request for url (http://[fd00:1122:3344:101::4]:32222/node/id): operation timed out: error sending request for url (http://[fd00:1122:3344:101::4]:32222/node/id): operation timed out: operation timed out","zone_id":"aa7c602b-6b2c-4ab0-8f78-6e74bc723578"}],"nsuccess":4}
12372024-08-22T17:50:25.066ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 5 reason = Timeout
12382024-08-22T17:50:25.067ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
12392024-08-22T17:50:25.067ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 5 reason = Timeout
12402024-08-22T17:50:25.067ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
12412024-08-22T17:50:25.067ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
12422024-08-22T17:50:25.067ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
12432024-08-22T17:50:25.069ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12442024-08-22T17:50:25.070ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
12452024-08-22T17:50:25.070ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
12462024-08-22T17:50:25.071ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.667104ms iteration = 5 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
12472024-08-22T17:50:25.071ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
12482024-08-22T17:50:25.071ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
12492024-08-22T17:50:25.072ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
12502024-08-22T17:50:25.074ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
12512024-08-22T17:50:25.074ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
12522024-08-22T17:50:25.075ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12532024-08-22T17:50:25.077ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 147bdc10-344c-4ee8-a219-603cfc66e6d2, lookup_type: ById(147bdc10-344c-4ee8-a219-603cfc66e6d2) } result = Ok(())
12542024-08-22T17:50:25.077ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
12552024-08-22T17:50:25.078ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
12562024-08-22T17:50:25.086ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 147bdc10-344c-4ee8-a219-603cfc66e6d2 original_time_created = 2024-08-22 17:49:18.445237 UTC target_id = 147bdc10-344c-4ee8-a219-603cfc66e6d2 time_created = 2024-08-22 17:49:18.445237 UTC
12572024-08-22T17:50:25.086ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 19.41638ms iteration = 5 status = {"status":"target blueprint unchanged","target_id":"147bdc10-344c-4ee8-a219-603cfc66e6d2","time_created":"2024-08-22T17:49:18.445237Z"}
12582024-08-22T17:50:35.067ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 6 reason = Timeout
12592024-08-22T17:50:35.067ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 6 reason = Timeout
12602024-08-22T17:50:35.067ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
12612024-08-22T17:50:35.067ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
12622024-08-22T17:50:35.067ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
12632024-08-22T17:50:35.067ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
12642024-08-22T17:50:35.069ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
12652024-08-22T17:50:35.069ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12662024-08-22T17:50:35.069ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
12672024-08-22T17:50:35.070ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.638559ms iteration = 6 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
12682024-08-22T17:50:35.071ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
12692024-08-22T17:50:35.071ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
12702024-08-22T17:50:35.071ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
12712024-08-22T17:50:35.073ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
12722024-08-22T17:50:35.073ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
12732024-08-22T17:50:35.075ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12742024-08-22T17:50:35.077ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 147bdc10-344c-4ee8-a219-603cfc66e6d2, lookup_type: ById(147bdc10-344c-4ee8-a219-603cfc66e6d2) } result = Ok(())
12752024-08-22T17:50:35.077ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
12762024-08-22T17:50:35.077ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
12772024-08-22T17:50:35.083ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 147bdc10-344c-4ee8-a219-603cfc66e6d2 original_time_created = 2024-08-22 17:49:18.445237 UTC target_id = 147bdc10-344c-4ee8-a219-603cfc66e6d2 time_created = 2024-08-22 17:49:18.445237 UTC
12782024-08-22T17:50:35.083ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 16.153984ms iteration = 6 status = {"status":"target blueprint unchanged","target_id":"147bdc10-344c-4ee8-a219-603cfc66e6d2","time_created":"2024-08-22T17:49:18.445237Z"}
12792024-08-22T17:50:45.067ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 2 reason = Timeout
12802024-08-22T17:50:45.067ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 2 reason = Timeout
12812024-08-22T17:50:45.067ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
12822024-08-22T17:50:45.067ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 2 reason = Timeout
12832024-08-22T17:50:45.067ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2024-08-22 17:40:45.067529190 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
12842024-08-22T17:50:45.067ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
12852024-08-22T17:50:45.067ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 2 reason = Timeout
12862024-08-22T17:50:45.067ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
12872024-08-22T17:50:45.067ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
12882024-08-22T17:50:45.067ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 3 reason = Timeout
12892024-08-22T17:50:45.067ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 2 reason = Timeout
12902024-08-22T17:50:45.067ZINFOnexus (BackgroundTasks): Abandoned VMM reaper running background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:76
12912024-08-22T17:50:45.067ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
12922024-08-22T17:50:45.067ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
12932024-08-22T17:50:45.067ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 4 reason = Timeout
12942024-08-22T17:50:45.067ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 3 reason = Timeout
12952024-08-22T17:50:45.067ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 2 reason = Timeout
12962024-08-22T17:50:45.067ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
12972024-08-22T17:50:45.067ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
12982024-08-22T17:50:45.067ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 3 reason = Timeout
12992024-08-22T17:50:45.067ZWARNnexus (BackgroundTasks): DNS propagation: skipped background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:69 reason = no config
13002024-08-22T17:50:45.067ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 9.951\u{b5}s iteration = 3 status = {"error":"no config"}
13012024-08-22T17:50:45.067ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 2 reason = Timeout
13022024-08-22T17:50:45.067ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
13032024-08-22T17:50:45.067ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
13042024-08-22T17:50:45.067ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 3 reason = Timeout
13052024-08-22T17:50:45.067ZWARNnexus (BackgroundTasks): DNS propagation: skipped background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:69 reason = no config
13062024-08-22T17:50:45.068ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 7 reason = Timeout
13072024-08-22T17:50:45.068ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 3 reason = Timeout
13082024-08-22T17:50:45.068ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 19.11\u{b5}s iteration = 3 status = {"error":"no config"}
13092024-08-22T17:50:45.068ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 2 reason = Timeout
13102024-08-22T17:50:45.068ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 3 reason = Timeout
13112024-08-22T17:50:45.068ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
13122024-08-22T17:50:45.068ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
13132024-08-22T17:50:45.068ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:80 reason = blueprint disabled target_id = 147bdc10-344c-4ee8-a219-603cfc66e6d2
13142024-08-22T17:50:45.068ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 3 reason = Timeout
13152024-08-22T17:50:45.068ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 3 reason = Timeout
13162024-08-22T17:50:45.068ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 20.09\u{b5}s iteration = 3 status = {"error":"blueprint disabled","target_id":"147bdc10-344c-4ee8-a219-603cfc66e6d2"}
13172024-08-22T17:50:45.068ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
13182024-08-22T17:50:45.068ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
13192024-08-22T17:50:45.068ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 2 reason = Timeout
13202024-08-22T17:50:45.068ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 3 reason = Timeout
13212024-08-22T17:50:45.068ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:56
13222024-08-22T17:50:45.068ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
13232024-08-22T17:50:45.068ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
13242024-08-22T17:50:45.068ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 3 reason = Timeout
13252024-08-22T17:50:45.068ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
13262024-08-22T17:50:45.068ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
13272024-08-22T17:50:45.068ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 3 reason = Timeout
13282024-08-22T17:50:45.068ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
13292024-08-22T17:50:45.068ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 3 reason = Timeout
13302024-08-22T17:50:45.068ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 7 reason = Timeout
13312024-08-22T17:50:45.068ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
13322024-08-22T17:50:45.068ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
13332024-08-22T17:50:45.068ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 3 reason = Timeout
13342024-08-22T17:50:45.068ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
13352024-08-22T17:50:45.068ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 3 reason = Timeout
13362024-08-22T17:50:45.068ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
13372024-08-22T17:50:45.068ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
13382024-08-22T17:50:45.068ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
13392024-08-22T17:50:45.068ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
13402024-08-22T17:50:45.068ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
13412024-08-22T17:50:45.068ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
13422024-08-22T17:50:45.068ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
13432024-08-22T17:50:45.068ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
13442024-08-22T17:50:45.068ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
13452024-08-22T17:50:45.068ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
13462024-08-22T17:50:45.068ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
13472024-08-22T17:50:45.068ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
13482024-08-22T17:50:45.068ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
13492024-08-22T17:50:45.068ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
13502024-08-22T17:50:45.068ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
13512024-08-22T17:50:45.068ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
13522024-08-22T17:50:45.068ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
13532024-08-22T17:50:45.068ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
13542024-08-22T17:50:45.068ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
13552024-08-22T17:50:45.068ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
13562024-08-22T17:50:45.068ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
13572024-08-22T17:50:45.069ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 3 reason = Timeout
13582024-08-22T17:50:45.069ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
13592024-08-22T17:50:45.069ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
13602024-08-22T17:50:45.070ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("9488ce0a-6732-4f75-baf9-4ecfa406df91.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("afac2eab-cadf-48d0-94a7-ad562ade88c6.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("afac2eab-cadf-48d0-94a7-ad562ade88c6.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::9))) }], valid_until: Instant { tv_sec: 1633, tv_nsec: 603786429 } })
13612024-08-22T17:50:45.070ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("6507c79b-9a18-46c7-930d-c98e8662b87f.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("a09ea923-b85c-428d-b289-6936a0c2542c.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("c0ef020c-52c1-4b9a-a085-445fab31264f.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("c0ef020c-52c1-4b9a-a085-445fab31264f.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:3::1))) }], valid_until: Instant { tv_sec: 1633, tv_nsec: 604020100 } })
13622024-08-22T17:50:45.070ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
13632024-08-22T17:50:45.070ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
13642024-08-22T17:50:45.071ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 4.248962ms iteration = 2 status = {"errors":[],"found_port_ok":[]}
13652024-08-22T17:50:45.071ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
13662024-08-22T17:50:45.072ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 4.416192ms iteration = 3 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
13672024-08-22T17:50:45.072ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2024-08-22 17:40:45.067529190 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
13682024-08-22T17:50:45.072ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 4.563674ms iteration = 2 status = {"expiration":"2024-08-22T17:40:45.067529190Z","pruned":[]}
13692024-08-22T17:50:45.072ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
13702024-08-22T17:50:45.073ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
13712024-08-22T17:50:45.073ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 5.709549ms iteration = 2 status = {"addresses":["[fd00:1122:3344:101::8]:5353","[fd00:1122:3344:101::9]:5353"]}
13722024-08-22T17:50:45.073ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13732024-08-22T17:50:45.074ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13742024-08-22T17:50:45.076ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
13752024-08-22T17:50:45.076ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
13762024-08-22T17:50:45.076ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
13772024-08-22T17:50:45.077ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } dns_group = internal
13782024-08-22T17:50:45.077ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 9.581029ms iteration = 2 status = {"addresses":["[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:3::1]:5353"]}
13792024-08-22T17:50:45.077ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
13802024-08-22T17:50:45.077ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
13812024-08-22T17:50:45.077ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
13822024-08-22T17:50:45.077ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-c961ede4-7532-4ca1-95a5-690b4d7c9b5a.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-c961ede4-7532-4ca1-95a5-690b4d7c9b5a.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1633, tv_nsec: 611475109 } })
13832024-08-22T17:50:45.079ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
13842024-08-22T17:50:45.079ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
13852024-08-22T17:50:45.079ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1050
13862024-08-22T17:50:45.080ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13872024-08-22T17:50:45.081ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13882024-08-22T17:50:45.082ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13892024-08-22T17:50:45.082ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
13902024-08-22T17:50:45.082ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
13912024-08-22T17:50:45.082ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
13922024-08-22T17:50:45.085ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
13932024-08-22T17:50:45.085ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
13942024-08-22T17:50:45.085ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
13952024-08-22T17:50:45.085ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
13962024-08-22T17:50:45.085ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
13972024-08-22T17:50:45.085ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
13982024-08-22T17:50:45.088ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13992024-08-22T17:50:45.088ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14002024-08-22T17:50:45.090ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
14012024-08-22T17:50:45.091ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
14022024-08-22T17:50:45.091ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
14032024-08-22T17:50:45.091ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
14042024-08-22T17:50:45.092ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
14052024-08-22T17:50:45.092ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
14062024-08-22T17:50:45.094ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
14072024-08-22T17:50:45.095ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14082024-08-22T17:50:45.095ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
14092024-08-22T17:50:45.097ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
14102024-08-22T17:50:45.097ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
14112024-08-22T17:50:45.097ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
14122024-08-22T17:50:45.099ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-c961ede4-7532-4ca1-95a5-690b4d7c9b5a.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-c961ede4-7532-4ca1-95a5-690b4d7c9b5a.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1633, tv_nsec: 633133061 } })
14132024-08-22T17:50:45.101ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1050
14142024-08-22T17:50:45.105ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:454 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
14152024-08-22T17:50:45.105ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 37.220552ms iteration = 3 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
14162024-08-22T17:50:45.108ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
14172024-08-22T17:50:45.108ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
14182024-08-22T17:50:45.124ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
14192024-08-22T17:50:45.124ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
14202024-08-22T17:50:45.125ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
14212024-08-22T17:50:45.126ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
14222024-08-22T17:50:45.128ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 60.520703ms iteration = 3 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
14232024-08-22T17:50:45.129ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
14242024-08-22T17:50:45.130ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
14252024-08-22T17:50:45.132ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0
14262024-08-22T17:50:45.132ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:189 found = 0 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0
14272024-08-22T17:50:45.132ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 64.443523ms iteration = 2 status = {"error":null,"error_count":0,"found":0,"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0}
14282024-08-22T17:50:45.132ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 63.431098ms iteration = 3 status = {"errors":[],"garbage_collect_requested":[]}
14292024-08-22T17:50:45.132ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14302024-08-22T17:50:45.134ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
14312024-08-22T17:50:45.134ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
14322024-08-22T17:50:45.134ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
14332024-08-22T17:50:45.135ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:55
14342024-08-22T17:50:45.135ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
14352024-08-22T17:50:45.135ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
14362024-08-22T17:50:45.136ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
14372024-08-22T17:50:45.137ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14382024-08-22T17:50:45.138ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-c961ede4-7532-4ca1-95a5-690b4d7c9b5a.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-c961ede4-7532-4ca1-95a5-690b4d7c9b5a.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1633, tv_nsec: 671735750 } })
14392024-08-22T17:50:45.138ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1050 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561
14402024-08-22T17:50:45.139ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
14412024-08-22T17:50:45.139ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
14422024-08-22T17:50:45.140ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
14432024-08-22T17:50:45.141ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14442024-08-22T17:50:45.142ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14452024-08-22T17:50:45.143ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
14462024-08-22T17:50:45.143ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
14472024-08-22T17:50:45.143ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 75.939782ms iteration = 7 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
14482024-08-22T17:50:45.143ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
14492024-08-22T17:50:45.146ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
14502024-08-22T17:50:45.146ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
14512024-08-22T17:50:45.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
14522024-08-22T17:50:45.148ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14532024-08-22T17:50:45.152ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
14542024-08-22T17:50:45.152ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
14552024-08-22T17:50:45.152ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
14562024-08-22T17:50:45.152ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:140 found = 0 not_ready_to_be_deleted = 0
14572024-08-22T17:50:45.152ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 84.835748ms iteration = 2 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
14582024-08-22T17:50:45.152ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
14592024-08-22T17:50:45.155ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14602024-08-22T17:50:45.157ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
14612024-08-22T17:50:45.157ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
14622024-08-22T17:50:45.158ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
14632024-08-22T17:50:45.158ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
14642024-08-22T17:50:45.158ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
14652024-08-22T17:50:45.161ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
14662024-08-22T17:50:45.161ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
14672024-08-22T17:50:45.161ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
14682024-08-22T17:50:45.161ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
14692024-08-22T17:50:45.161ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
14702024-08-22T17:50:45.162ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14712024-08-22T17:50:45.162ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 94.436559ms iteration = 4 status = {"physical_disks_added":0}
14722024-08-22T17:50:45.163ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14732024-08-22T17:50:45.164ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
14742024-08-22T17:50:45.164ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
14752024-08-22T17:50:45.164ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
14762024-08-22T17:50:45.167ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
14772024-08-22T17:50:45.167ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
14782024-08-22T17:50:45.167ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
14792024-08-22T17:50:45.180ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 112.271401ms iteration = 3 status = {"region_replacement_started_err":0,"region_replacement_started_ok":0}
14802024-08-22T17:50:45.183ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:55
14812024-08-22T17:50:45.183ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:220 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
14822024-08-22T17:50:45.183ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 115.639648ms iteration = 3 status = {"destroyed_active_vmms":0,"error":null,"saga_start_failures":0,"sagas_completed":0,"sagas_failed":0,"sagas_started":0,"terminated_active_migrations":0}
14832024-08-22T17:50:45.188ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1058 zone_address = fd00:1122:3344:101::2
14842024-08-22T17:50:45.188ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
14852024-08-22T17:50:45.188ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "4d64be47-2c31-457a-b230-57a759a3b4da", "content-length": "26", "date": "Thu, 22 Aug 2024 17:50:44 GMT"} })
14862024-08-22T17:50:45.188ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1063 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
14872024-08-22T17:50:45.190ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1095 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
14882024-08-22T17:50:45.194ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
14892024-08-22T17:50:45.194ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
14902024-08-22T17:50:45.198ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14912024-08-22T17:50:45.201ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 147bdc10-344c-4ee8-a219-603cfc66e6d2, lookup_type: ById(147bdc10-344c-4ee8-a219-603cfc66e6d2) } result = Ok(())
14922024-08-22T17:50:45.201ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
14932024-08-22T17:50:45.201ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
14942024-08-22T17:50:45.225ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1058 zone_address = fd00:1122:3344:101::2
14952024-08-22T17:50:45.225ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
14962024-08-22T17:50:45.225ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "cdbeba3e-ea67-493c-9135-aba47b06d1e6", "content-length": "26", "date": "Thu, 22 Aug 2024 17:50:44 GMT"} })
14972024-08-22T17:50:45.225ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1063 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
14982024-08-22T17:50:45.227ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1095 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
14992024-08-22T17:50:45.248ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
15002024-08-22T17:50:45.248ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
15012024-08-22T17:50:45.251ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
15022024-08-22T17:50:45.251ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
15032024-08-22T17:50:45.251ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
15042024-08-22T17:50:45.251ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
15052024-08-22T17:50:45.252ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
15062024-08-22T17:50:45.343ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1058 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 zone_address = fd00:1122:3344:101::2
15072024-08-22T17:50:45.343ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
15082024-08-22T17:50:45.344ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "f1989d9a-e477-4f99-9528-c0b017f7f1e0", "content-length": "26", "date": "Thu, 22 Aug 2024 17:50:44 GMT"} })
15092024-08-22T17:50:45.344ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1063 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
15102024-08-22T17:50:45.345ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1095 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561
15112024-08-22T17:50:45.365ZDEBGnexus (BackgroundTasks): client request SledAgent = c961ede4-7532-4ca1-95a5-690b4d7c9b5a background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
15122024-08-22T17:50:45.366ZDEBGnexus (BackgroundTasks): client response SledAgent = c961ede4-7532-4ca1-95a5-690b4d7c9b5a background_task = v2p_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/v2p", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "6169a4a2-1f40-43f6-9d26-dcbd694a509a", "content-length": "691", "date": "Thu, 22 Aug 2024 17:50:44 GMT"} })
15132024-08-22T17:50:45.366ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = buskin
15142024-08-22T17:50:45.366ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = buskin
15152024-08-22T17:50:45.366ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = buskin
15162024-08-22T17:50:45.367ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 299.772308ms iteration = 3 status = {}
15172024-08-22T17:50:45.368ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15182024-08-22T17:50:45.369ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15192024-08-22T17:50:45.369ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
15202024-08-22T17:50:45.370ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
15212024-08-22T17:50:45.370ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
15222024-08-22T17:50:45.370ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
15232024-08-22T17:50:45.370ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
15242024-08-22T17:50:45.372ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
15252024-08-22T17:50:45.372ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
15262024-08-22T17:50:45.372ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
15272024-08-22T17:50:45.377ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 147bdc10-344c-4ee8-a219-603cfc66e6d2 original_time_created = 2024-08-22 17:49:18.445237 UTC target_id = 147bdc10-344c-4ee8-a219-603cfc66e6d2 time_created = 2024-08-22 17:49:18.445237 UTC
15282024-08-22T17:50:45.377ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 309.455758ms iteration = 7 status = {"status":"target blueprint unchanged","target_id":"147bdc10-344c-4ee8-a219-603cfc66e6d2","time_created":"2024-08-22T17:49:18.445237Z"}
15292024-08-22T17:50:45.394ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
15302024-08-22T17:50:45.395ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "4451e215-4edf-4cdb-aa0f-8d3c074ae4af", "content-length": "1", "date": "Thu, 22 Aug 2024 17:50:45 GMT"} })
15312024-08-22T17:50:45.397ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
15322024-08-22T17:50:45.397ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
15332024-08-22T17:50:45.399ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 331.992665ms iteration = 3 status = null
15342024-08-22T17:50:45.401ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 333.882405ms iteration = 3 status = {}
15352024-08-22T17:50:45.401ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
15362024-08-22T17:50:45.401ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 7f16ce95-fbde-4896-9a22-7f61d8d01c5d dns_zone_name = oxide.test found_so_far = 0 version = 2
15372024-08-22T17:50:45.401ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
15382024-08-22T17:50:45.401ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
15392024-08-22T17:50:45.403ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15402024-08-22T17:50:45.406ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
15412024-08-22T17:50:45.406ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
15422024-08-22T17:50:45.406ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
15432024-08-22T17:50:45.406ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
15442024-08-22T17:50:45.406ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = d6f65b4e-2e87-480a-af21-a69b4520828c dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
15452024-08-22T17:50:45.406ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
15462024-08-22T17:50:45.406ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
15472024-08-22T17:50:45.407ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15482024-08-22T17:50:45.408ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15492024-08-22T17:50:45.410ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: c961ede4-7532-4ca1-95a5-690b4d7c9b5a, lookup_type: ById(c961ede4-7532-4ca1-95a5-690b4d7c9b5a) } result = Ok(())
15502024-08-22T17:50:45.410ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
15512024-08-22T17:50:45.410ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 7f16ce95-fbde-4896-9a22-7f61d8d01c5d dns_zone_name = oxide.test found_so_far = 1 version = 2
15522024-08-22T17:50:45.410ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
15532024-08-22T17:50:45.410ZINFOnexus (BackgroundTasks): found latest generation (first find) background_task = dns_config_external dns_group = external file = nexus/src/app/background/tasks/dns_config.rs:87 generation = 2
15542024-08-22T17:50:45.410ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 342.605999ms iteration = 2 status = {"generation":2}
15552024-08-22T17:50:45.410ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 4 reason = Dependency
15562024-08-22T17:50:45.410ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
15572024-08-22T17:50:45.412ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 344.54272ms iteration = 3 status = 0
15582024-08-22T17:50:45.412ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
15592024-08-22T17:50:45.412ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
15602024-08-22T17:50:45.412ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
15612024-08-22T17:50:45.416ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = d6f65b4e-2e87-480a-af21-a69b4520828c dns_zone_name = control-plane.oxide.internal found_so_far = 50 version = 1
15622024-08-22T17:50:45.416ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
15632024-08-22T17:50:45.416ZINFOnexus (BackgroundTasks): found latest generation (first find) background_task = dns_config_internal dns_group = internal file = nexus/src/app/background/tasks/dns_config.rs:87 generation = 1
15642024-08-22T17:50:45.416ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 348.575481ms iteration = 2 status = {"generation":1}
15652024-08-22T17:50:45.416ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 4 reason = Dependency
15662024-08-22T17:50:45.438ZINFOnexus (BackgroundTasks): found Silo/DNS/TLS config (initial) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-08-22T17:49:43.060798Z, time_modified: 2024-08-22T17:49:43.060798Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 10d39b39-43d3-41db-b622-3ef3e09e2dad, db_silo: Silo { identity: SiloIdentity { id: 10d39b39-43d3-41db-b622-3ef3e09e2dad, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-08-22T17:49:58.442383Z, time_modified: 2024-08-22T17:49:58.442383Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "852177ddd7117113b778290a63910551c7b25a6718e9c8fda4590027c9443fee" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 10d39b39-43d3-41db-b622-3ef3e09e2dad, db_silo: Silo { identity: SiloIdentity { id: 10d39b39-43d3-41db-b622-3ef3e09e2dad, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-08-22T17:49:58.442383Z, time_modified: 2024-08-22T17:49:58.442383Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "852177ddd7117113b778290a63910551c7b25a6718e9c8fda4590027c9443fee" }] }) } file = nexus/src/app/background/tasks/external_endpoints.rs:77
15672024-08-22T17:50:45.438ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 371.119846ms iteration = 2 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"10d39b39-43d3-41db-b622-3ef3e09e2dad","tls_certs":["852177ddd7117113b778290a63910551c7b25a6718e9c8fda4590027c9443fee"]}},"default_endpoint":{"silo_id":"10d39b39-43d3-41db-b622-3ef3e09e2dad","tls_certs":["852177ddd7117113b778290a63910551c7b25a6718e9c8fda4590027c9443fee"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
15682024-08-22T17:50:45.501ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /
15692024-08-22T17:50:45.501ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /access
15702024-08-22T17:50:45.501ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /assets/{path}
15712024-08-22T17:50:45.501ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /device/auth
15722024-08-22T17:50:45.501ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /device/confirm
15732024-08-22T17:50:45.501ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /device/success
15742024-08-22T17:50:45.501ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /device/token
15752024-08-22T17:50:45.501ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /device/verify
15762024-08-22T17:50:45.501ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /experimental/v1/probes
15772024-08-22T17:50:45.501ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /experimental/v1/probes
15782024-08-22T17:50:45.501ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /experimental/v1/probes/{probe}
15792024-08-22T17:50:45.501ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /experimental/v1/probes/{probe}
15802024-08-22T17:50:45.501ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /images
15812024-08-22T17:50:45.501ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /login
15822024-08-22T17:50:45.501ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /login/{silo_name}/local
15832024-08-22T17:50:45.501ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /login/{silo_name}/saml/{provider_name}
15842024-08-22T17:50:45.501ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /login/{silo_name}/saml/{provider_name}
15852024-08-22T17:50:45.501ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /login/{silo_name}/saml/{provider_name}/redirect
15862024-08-22T17:50:45.501ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /lookup/{path}
15872024-08-22T17:50:45.501ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /projects/{path}
15882024-08-22T17:50:45.501ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /projects-new
15892024-08-22T17:50:45.501ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /settings/{path}
15902024-08-22T17:50:45.501ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /system/{path}
15912024-08-22T17:50:45.501ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /utilization
15922024-08-22T17:50:45.501ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/certificates
15932024-08-22T17:50:45.501ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/certificates
15942024-08-22T17:50:45.501ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/certificates/{certificate}
15952024-08-22T17:50:45.501ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/certificates/{certificate}
15962024-08-22T17:50:45.501ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/disks
15972024-08-22T17:50:45.501ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/disks
15982024-08-22T17:50:45.501ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/disks/{disk}
15992024-08-22T17:50:45.501ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/disks/{disk}
16002024-08-22T17:50:45.501ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/disks/{disk}/bulk-write
16012024-08-22T17:50:45.501ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/disks/{disk}/bulk-write-start
16022024-08-22T17:50:45.501ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/disks/{disk}/bulk-write-stop
16032024-08-22T17:50:45.501ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/disks/{disk}/finalize
16042024-08-22T17:50:45.501ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/disks/{disk}/metrics/{metric}
16052024-08-22T17:50:45.501ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/floating-ips
16062024-08-22T17:50:45.501ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/floating-ips
16072024-08-22T17:50:45.501ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/floating-ips/{floating_ip}
16082024-08-22T17:50:45.501ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/floating-ips/{floating_ip}
16092024-08-22T17:50:45.501ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = PUT path = /v1/floating-ips/{floating_ip}
16102024-08-22T17:50:45.501ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/floating-ips/{floating_ip}/attach
16112024-08-22T17:50:45.501ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/floating-ips/{floating_ip}/detach
16122024-08-22T17:50:45.501ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/groups
16132024-08-22T17:50:45.501ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/groups/{group_id}
16142024-08-22T17:50:45.501ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/images
16152024-08-22T17:50:45.501ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/images
16162024-08-22T17:50:45.501ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/images/{image}
16172024-08-22T17:50:45.501ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/images/{image}
16182024-08-22T17:50:45.501ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/images/{image}/demote
16192024-08-22T17:50:45.501ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/images/{image}/promote
16202024-08-22T17:50:45.501ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/instances
16212024-08-22T17:50:45.501ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/instances
16222024-08-22T17:50:45.501ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/instances/{instance}
16232024-08-22T17:50:45.501ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/instances/{instance}
16242024-08-22T17:50:45.501ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/instances/{instance}/disks
16252024-08-22T17:50:45.501ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/instances/{instance}/disks/attach
16262024-08-22T17:50:45.501ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/instances/{instance}/disks/detach
16272024-08-22T17:50:45.501ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/instances/{instance}/external-ips
16282024-08-22T17:50:45.501ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/instances/{instance}/external-ips/ephemeral
16292024-08-22T17:50:45.501ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/instances/{instance}/external-ips/ephemeral
16302024-08-22T17:50:45.501ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/instances/{instance}/reboot
16312024-08-22T17:50:45.501ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/instances/{instance}/serial-console
16322024-08-22T17:50:45.501ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/instances/{instance}/serial-console/stream
16332024-08-22T17:50:45.501ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/instances/{instance}/ssh-public-keys
16342024-08-22T17:50:45.501ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/instances/{instance}/start
16352024-08-22T17:50:45.501ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/instances/{instance}/stop
16362024-08-22T17:50:45.501ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/ip-pools
16372024-08-22T17:50:45.501ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/ip-pools/{pool}
16382024-08-22T17:50:45.501ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/login/{silo_name}/local
16392024-08-22T17:50:45.501ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/logout
16402024-08-22T17:50:45.501ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/me
16412024-08-22T17:50:45.501ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/me/groups
16422024-08-22T17:50:45.501ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/me/ssh-keys
16432024-08-22T17:50:45.501ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/me/ssh-keys
16442024-08-22T17:50:45.501ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/me/ssh-keys/{ssh_key}
16452024-08-22T17:50:45.501ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/me/ssh-keys/{ssh_key}
16462024-08-22T17:50:45.501ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/metrics/{metric_name}
16472024-08-22T17:50:45.501ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/network-interfaces
16482024-08-22T17:50:45.501ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/network-interfaces
16492024-08-22T17:50:45.501ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/network-interfaces/{interface}
16502024-08-22T17:50:45.501ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/network-interfaces/{interface}
16512024-08-22T17:50:45.501ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = PUT path = /v1/network-interfaces/{interface}
16522024-08-22T17:50:45.501ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/ping
16532024-08-22T17:50:45.501ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/policy
16542024-08-22T17:50:45.501ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = PUT path = /v1/policy
16552024-08-22T17:50:45.501ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/projects
16562024-08-22T17:50:45.501ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/projects
16572024-08-22T17:50:45.501ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/projects/{project}
16582024-08-22T17:50:45.501ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/projects/{project}
16592024-08-22T17:50:45.501ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = PUT path = /v1/projects/{project}
16602024-08-22T17:50:45.501ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/projects/{project}/policy
16612024-08-22T17:50:45.501ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = PUT path = /v1/projects/{project}/policy
16622024-08-22T17:50:45.501ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/snapshots
16632024-08-22T17:50:45.501ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/snapshots
16642024-08-22T17:50:45.501ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/snapshots/{snapshot}
16652024-08-22T17:50:45.501ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/snapshots/{snapshot}
16662024-08-22T17:50:45.502ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/hardware/disks
16672024-08-22T17:50:45.502ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/hardware/disks/{disk_id}
16682024-08-22T17:50:45.502ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/hardware/racks
16692024-08-22T17:50:45.502ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/hardware/racks/{rack_id}
16702024-08-22T17:50:45.502ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/hardware/sleds
16712024-08-22T17:50:45.502ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/system/hardware/sleds
16722024-08-22T17:50:45.502ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/hardware/sleds/{sled_id}
16732024-08-22T17:50:45.502ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/hardware/sleds/{sled_id}/disks
16742024-08-22T17:50:45.502ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/hardware/sleds/{sled_id}/instances
16752024-08-22T17:50:45.502ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = PUT path = /v1/system/hardware/sleds/{sled_id}/provision-policy
16762024-08-22T17:50:45.502ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/hardware/sleds-uninitialized
16772024-08-22T17:50:45.502ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/hardware/switch-port
16782024-08-22T17:50:45.502ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/system/hardware/switch-port/{port}/settings
16792024-08-22T17:50:45.502ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/system/hardware/switch-port/{port}/settings
16802024-08-22T17:50:45.502ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/hardware/switch-port/{port}/status
16812024-08-22T17:50:45.502ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/hardware/switches
16822024-08-22T17:50:45.502ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/hardware/switches/{switch_id}
16832024-08-22T17:50:45.502ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/identity-providers
16842024-08-22T17:50:45.502ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/system/identity-providers/local/users
16852024-08-22T17:50:45.502ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/system/identity-providers/local/users/{user_id}
16862024-08-22T17:50:45.502ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/system/identity-providers/local/users/{user_id}/set-password
16872024-08-22T17:50:45.502ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/system/identity-providers/saml
16882024-08-22T17:50:45.502ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/identity-providers/saml/{provider}
16892024-08-22T17:50:45.502ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/ip-pools
16902024-08-22T17:50:45.502ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/system/ip-pools
16912024-08-22T17:50:45.502ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/system/ip-pools/{pool}
16922024-08-22T17:50:45.502ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/ip-pools/{pool}
16932024-08-22T17:50:45.502ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = PUT path = /v1/system/ip-pools/{pool}
16942024-08-22T17:50:45.502ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/ip-pools/{pool}/ranges
16952024-08-22T17:50:45.502ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/system/ip-pools/{pool}/ranges/add
16962024-08-22T17:50:45.502ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/system/ip-pools/{pool}/ranges/remove
16972024-08-22T17:50:45.502ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/ip-pools/{pool}/silos
16982024-08-22T17:50:45.502ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/system/ip-pools/{pool}/silos
16992024-08-22T17:50:45.502ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/system/ip-pools/{pool}/silos/{silo}
17002024-08-22T17:50:45.502ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = PUT path = /v1/system/ip-pools/{pool}/silos/{silo}
17012024-08-22T17:50:45.502ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/ip-pools/{pool}/utilization
17022024-08-22T17:50:45.502ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/ip-pools-service
17032024-08-22T17:50:45.502ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/ip-pools-service/ranges
17042024-08-22T17:50:45.502ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/system/ip-pools-service/ranges/add
17052024-08-22T17:50:45.502ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/system/ip-pools-service/ranges/remove
17062024-08-22T17:50:45.502ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/metrics/{metric_name}
17072024-08-22T17:50:45.502ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/networking/address-lot
17082024-08-22T17:50:45.502ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/system/networking/address-lot
17092024-08-22T17:50:45.502ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/system/networking/address-lot/{address_lot}
17102024-08-22T17:50:45.502ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/networking/address-lot/{address_lot}/blocks
17112024-08-22T17:50:45.502ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/networking/allow-list
17122024-08-22T17:50:45.502ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = PUT path = /v1/system/networking/allow-list
17132024-08-22T17:50:45.502ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/system/networking/bfd-disable
17142024-08-22T17:50:45.502ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/system/networking/bfd-enable
17152024-08-22T17:50:45.502ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/networking/bfd-status
17162024-08-22T17:50:45.502ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/system/networking/bgp
17172024-08-22T17:50:45.502ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/networking/bgp
17182024-08-22T17:50:45.502ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/system/networking/bgp
17192024-08-22T17:50:45.502ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/networking/bgp-announce-set
17202024-08-22T17:50:45.502ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = PUT path = /v1/system/networking/bgp-announce-set
17212024-08-22T17:50:45.502ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/system/networking/bgp-announce-set/{name_or_id}
17222024-08-22T17:50:45.502ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/networking/bgp-announce-set/{name_or_id}/announcement
17232024-08-22T17:50:45.502ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/networking/bgp-exported
17242024-08-22T17:50:45.502ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/networking/bgp-message-history
17252024-08-22T17:50:45.502ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/networking/bgp-routes-ipv4
17262024-08-22T17:50:45.502ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/networking/bgp-status
17272024-08-22T17:50:45.502ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/networking/loopback-address
17282024-08-22T17:50:45.502ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/system/networking/loopback-address
17292024-08-22T17:50:45.502ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/system/networking/loopback-address/{rack_id}/{switch_location}/{address}/{subnet_mask}
17302024-08-22T17:50:45.502ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/system/networking/switch-port-settings
17312024-08-22T17:50:45.502ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/networking/switch-port-settings
17322024-08-22T17:50:45.502ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/system/networking/switch-port-settings
17332024-08-22T17:50:45.502ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/networking/switch-port-settings/{port}
17342024-08-22T17:50:45.502ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/policy
17352024-08-22T17:50:45.502ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = PUT path = /v1/system/policy
17362024-08-22T17:50:45.502ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/roles
17372024-08-22T17:50:45.502ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/roles/{role_name}
17382024-08-22T17:50:45.502ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/silo-quotas
17392024-08-22T17:50:45.502ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/silos
17402024-08-22T17:50:45.502ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/system/silos
17412024-08-22T17:50:45.502ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/system/silos/{silo}
17422024-08-22T17:50:45.502ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/silos/{silo}
17432024-08-22T17:50:45.502ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/silos/{silo}/ip-pools
17442024-08-22T17:50:45.502ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/silos/{silo}/policy
17452024-08-22T17:50:45.502ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = PUT path = /v1/system/silos/{silo}/policy
17462024-08-22T17:50:45.502ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/silos/{silo}/quotas
17472024-08-22T17:50:45.502ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = PUT path = /v1/system/silos/{silo}/quotas
17482024-08-22T17:50:45.502ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = PUT path = /v1/system/update/repository
17492024-08-22T17:50:45.502ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/update/repository/{system_version}
17502024-08-22T17:50:45.502ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/users
17512024-08-22T17:50:45.502ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/users/{user_id}
17522024-08-22T17:50:45.502ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/users-builtin
17532024-08-22T17:50:45.502ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/users-builtin/{user}
17542024-08-22T17:50:45.502ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/utilization/silos
17552024-08-22T17:50:45.502ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/utilization/silos/{silo}
17562024-08-22T17:50:45.502ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/timeseries/query
17572024-08-22T17:50:45.502ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/timeseries/schema
17582024-08-22T17:50:45.502ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/users
17592024-08-22T17:50:45.502ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/utilization
17602024-08-22T17:50:45.502ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/vpc-firewall-rules
17612024-08-22T17:50:45.502ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = PUT path = /v1/vpc-firewall-rules
17622024-08-22T17:50:45.502ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/vpc-router-routes
17632024-08-22T17:50:45.502ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/vpc-router-routes
17642024-08-22T17:50:45.502ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/vpc-router-routes/{route}
17652024-08-22T17:50:45.502ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/vpc-router-routes/{route}
17662024-08-22T17:50:45.502ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = PUT path = /v1/vpc-router-routes/{route}
17672024-08-22T17:50:45.502ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/vpc-routers
17682024-08-22T17:50:45.502ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/vpc-routers
17692024-08-22T17:50:45.502ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/vpc-routers/{router}
17702024-08-22T17:50:45.502ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/vpc-routers/{router}
17712024-08-22T17:50:45.502ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = PUT path = /v1/vpc-routers/{router}
17722024-08-22T17:50:45.502ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/vpc-subnets
17732024-08-22T17:50:45.502ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/vpc-subnets
17742024-08-22T17:50:45.502ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/vpc-subnets/{subnet}
17752024-08-22T17:50:45.502ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/vpc-subnets/{subnet}
17762024-08-22T17:50:45.502ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = PUT path = /v1/vpc-subnets/{subnet}
17772024-08-22T17:50:45.502ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/vpc-subnets/{subnet}/network-interfaces
17782024-08-22T17:50:45.503ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/vpcs
17792024-08-22T17:50:45.503ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/vpcs
17802024-08-22T17:50:45.503ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/vpcs/{vpc}
17812024-08-22T17:50:45.503ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/vpcs/{vpc}
17822024-08-22T17:50:45.503ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = PUT path = /v1/vpcs/{vpc}
17832024-08-22T17:50:45.503ZINFOnexus (dropshot_external): listening file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:205 local_addr = 172.30.2.7:443
17842024-08-22T17:50:45.503ZDEBGnexus (dropshot_external): successfully registered DTrace USDT probes local_addr = 172.30.2.7:443
17852024-08-22T17:50:45.509ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /
17862024-08-22T17:50:45.509ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /access
17872024-08-22T17:50:45.509ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /assets/{path}
17882024-08-22T17:50:45.509ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /device/auth
17892024-08-22T17:50:45.509ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /device/confirm
17902024-08-22T17:50:45.509ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /device/success
17912024-08-22T17:50:45.509ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /device/token
17922024-08-22T17:50:45.509ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /device/verify
17932024-08-22T17:50:45.509ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /experimental/v1/probes
17942024-08-22T17:50:45.509ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /experimental/v1/probes
17952024-08-22T17:50:45.509ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /experimental/v1/probes/{probe}
17962024-08-22T17:50:45.509ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /experimental/v1/probes/{probe}
17972024-08-22T17:50:45.509ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /images
17982024-08-22T17:50:45.509ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /login
17992024-08-22T17:50:45.509ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /login/{silo_name}/local
18002024-08-22T17:50:45.509ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /login/{silo_name}/saml/{provider_name}
18012024-08-22T17:50:45.509ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /login/{silo_name}/saml/{provider_name}
18022024-08-22T17:50:45.509ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /login/{silo_name}/saml/{provider_name}/redirect
18032024-08-22T17:50:45.509ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /lookup/{path}
18042024-08-22T17:50:45.509ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /projects/{path}
18052024-08-22T17:50:45.509ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /projects-new
18062024-08-22T17:50:45.509ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /settings/{path}
18072024-08-22T17:50:45.509ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /system/{path}
18082024-08-22T17:50:45.509ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /utilization
18092024-08-22T17:50:45.509ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/certificates
18102024-08-22T17:50:45.509ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/certificates
18112024-08-22T17:50:45.509ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/certificates/{certificate}
18122024-08-22T17:50:45.509ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/certificates/{certificate}
18132024-08-22T17:50:45.509ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/disks
18142024-08-22T17:50:45.509ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/disks
18152024-08-22T17:50:45.509ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/disks/{disk}
18162024-08-22T17:50:45.509ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/disks/{disk}
18172024-08-22T17:50:45.509ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/disks/{disk}/bulk-write
18182024-08-22T17:50:45.509ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/disks/{disk}/bulk-write-start
18192024-08-22T17:50:45.509ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/disks/{disk}/bulk-write-stop
18202024-08-22T17:50:45.509ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/disks/{disk}/finalize
18212024-08-22T17:50:45.509ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/disks/{disk}/metrics/{metric}
18222024-08-22T17:50:45.509ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/floating-ips
18232024-08-22T17:50:45.509ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/floating-ips
18242024-08-22T17:50:45.509ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/floating-ips/{floating_ip}
18252024-08-22T17:50:45.509ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/floating-ips/{floating_ip}
18262024-08-22T17:50:45.509ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = PUT path = /v1/floating-ips/{floating_ip}
18272024-08-22T17:50:45.509ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/floating-ips/{floating_ip}/attach
18282024-08-22T17:50:45.509ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/floating-ips/{floating_ip}/detach
18292024-08-22T17:50:45.509ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/groups
18302024-08-22T17:50:45.509ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/groups/{group_id}
18312024-08-22T17:50:45.509ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/images
18322024-08-22T17:50:45.509ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/images
18332024-08-22T17:50:45.509ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/images/{image}
18342024-08-22T17:50:45.509ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/images/{image}
18352024-08-22T17:50:45.509ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/images/{image}/demote
18362024-08-22T17:50:45.509ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/images/{image}/promote
18372024-08-22T17:50:45.509ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/instances
18382024-08-22T17:50:45.509ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/instances
18392024-08-22T17:50:45.509ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/instances/{instance}
18402024-08-22T17:50:45.509ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/instances/{instance}
18412024-08-22T17:50:45.509ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/instances/{instance}/disks
18422024-08-22T17:50:45.509ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/instances/{instance}/disks/attach
18432024-08-22T17:50:45.509ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/instances/{instance}/disks/detach
18442024-08-22T17:50:45.509ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/instances/{instance}/external-ips
18452024-08-22T17:50:45.509ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/instances/{instance}/external-ips/ephemeral
18462024-08-22T17:50:45.509ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/instances/{instance}/external-ips/ephemeral
18472024-08-22T17:50:45.509ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/instances/{instance}/reboot
18482024-08-22T17:50:45.509ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/instances/{instance}/serial-console
18492024-08-22T17:50:45.509ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/instances/{instance}/serial-console/stream
18502024-08-22T17:50:45.509ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/instances/{instance}/ssh-public-keys
18512024-08-22T17:50:45.509ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/instances/{instance}/start
18522024-08-22T17:50:45.509ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/instances/{instance}/stop
18532024-08-22T17:50:45.509ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/ip-pools
18542024-08-22T17:50:45.509ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/ip-pools/{pool}
18552024-08-22T17:50:45.509ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/login/{silo_name}/local
18562024-08-22T17:50:45.509ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/logout
18572024-08-22T17:50:45.509ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/me
18582024-08-22T17:50:45.509ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/me/groups
18592024-08-22T17:50:45.509ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/me/ssh-keys
18602024-08-22T17:50:45.509ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/me/ssh-keys
18612024-08-22T17:50:45.509ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/me/ssh-keys/{ssh_key}
18622024-08-22T17:50:45.509ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/me/ssh-keys/{ssh_key}
18632024-08-22T17:50:45.509ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/metrics/{metric_name}
18642024-08-22T17:50:45.509ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/network-interfaces
18652024-08-22T17:50:45.509ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/network-interfaces
18662024-08-22T17:50:45.509ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/network-interfaces/{interface}
18672024-08-22T17:50:45.509ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/network-interfaces/{interface}
18682024-08-22T17:50:45.509ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = PUT path = /v1/network-interfaces/{interface}
18692024-08-22T17:50:45.509ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/ping
18702024-08-22T17:50:45.509ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/policy
18712024-08-22T17:50:45.509ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = PUT path = /v1/policy
18722024-08-22T17:50:45.509ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/projects
18732024-08-22T17:50:45.509ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/projects
18742024-08-22T17:50:45.509ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/projects/{project}
18752024-08-22T17:50:45.510ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/projects/{project}
18762024-08-22T17:50:45.510ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = PUT path = /v1/projects/{project}
18772024-08-22T17:50:45.510ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/projects/{project}/policy
18782024-08-22T17:50:45.510ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = PUT path = /v1/projects/{project}/policy
18792024-08-22T17:50:45.510ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/snapshots
18802024-08-22T17:50:45.510ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/snapshots
18812024-08-22T17:50:45.510ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/snapshots/{snapshot}
18822024-08-22T17:50:45.510ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/snapshots/{snapshot}
18832024-08-22T17:50:45.510ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/hardware/disks
18842024-08-22T17:50:45.510ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/hardware/disks/{disk_id}
18852024-08-22T17:50:45.510ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/hardware/racks
18862024-08-22T17:50:45.510ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/hardware/racks/{rack_id}
18872024-08-22T17:50:45.510ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/hardware/sleds
18882024-08-22T17:50:45.510ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/system/hardware/sleds
18892024-08-22T17:50:45.510ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/hardware/sleds/{sled_id}
18902024-08-22T17:50:45.510ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/hardware/sleds/{sled_id}/disks
18912024-08-22T17:50:45.510ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/hardware/sleds/{sled_id}/instances
18922024-08-22T17:50:45.510ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = PUT path = /v1/system/hardware/sleds/{sled_id}/provision-policy
18932024-08-22T17:50:45.510ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/hardware/sleds-uninitialized
18942024-08-22T17:50:45.510ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/hardware/switch-port
18952024-08-22T17:50:45.510ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/system/hardware/switch-port/{port}/settings
18962024-08-22T17:50:45.510ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/system/hardware/switch-port/{port}/settings
18972024-08-22T17:50:45.510ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/hardware/switch-port/{port}/status
18982024-08-22T17:50:45.510ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/hardware/switches
18992024-08-22T17:50:45.510ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/hardware/switches/{switch_id}
19002024-08-22T17:50:45.510ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/identity-providers
19012024-08-22T17:50:45.510ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/system/identity-providers/local/users
19022024-08-22T17:50:45.510ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/system/identity-providers/local/users/{user_id}
19032024-08-22T17:50:45.510ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/system/identity-providers/local/users/{user_id}/set-password
19042024-08-22T17:50:45.510ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/system/identity-providers/saml
19052024-08-22T17:50:45.510ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/identity-providers/saml/{provider}
19062024-08-22T17:50:45.510ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/ip-pools
19072024-08-22T17:50:45.510ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/system/ip-pools
19082024-08-22T17:50:45.510ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/system/ip-pools/{pool}
19092024-08-22T17:50:45.510ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/ip-pools/{pool}
19102024-08-22T17:50:45.510ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = PUT path = /v1/system/ip-pools/{pool}
19112024-08-22T17:50:45.510ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/ip-pools/{pool}/ranges
19122024-08-22T17:50:45.510ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/system/ip-pools/{pool}/ranges/add
19132024-08-22T17:50:45.510ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/system/ip-pools/{pool}/ranges/remove
19142024-08-22T17:50:45.510ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/ip-pools/{pool}/silos
19152024-08-22T17:50:45.510ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/system/ip-pools/{pool}/silos
19162024-08-22T17:50:45.510ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/system/ip-pools/{pool}/silos/{silo}
19172024-08-22T17:50:45.510ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = PUT path = /v1/system/ip-pools/{pool}/silos/{silo}
19182024-08-22T17:50:45.510ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/ip-pools/{pool}/utilization
19192024-08-22T17:50:45.510ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/ip-pools-service
19202024-08-22T17:50:45.510ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/ip-pools-service/ranges
19212024-08-22T17:50:45.510ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/system/ip-pools-service/ranges/add
19222024-08-22T17:50:45.510ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/system/ip-pools-service/ranges/remove
19232024-08-22T17:50:45.510ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/metrics/{metric_name}
19242024-08-22T17:50:45.510ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/networking/address-lot
19252024-08-22T17:50:45.510ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/system/networking/address-lot
19262024-08-22T17:50:45.510ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/system/networking/address-lot/{address_lot}
19272024-08-22T17:50:45.510ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/networking/address-lot/{address_lot}/blocks
19282024-08-22T17:50:45.510ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/networking/allow-list
19292024-08-22T17:50:45.510ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = PUT path = /v1/system/networking/allow-list
19302024-08-22T17:50:45.510ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/system/networking/bfd-disable
19312024-08-22T17:50:45.510ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/system/networking/bfd-enable
19322024-08-22T17:50:45.510ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/networking/bfd-status
19332024-08-22T17:50:45.510ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/system/networking/bgp
19342024-08-22T17:50:45.510ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/networking/bgp
19352024-08-22T17:50:45.510ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/system/networking/bgp
19362024-08-22T17:50:45.510ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/networking/bgp-announce-set
19372024-08-22T17:50:45.510ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = PUT path = /v1/system/networking/bgp-announce-set
19382024-08-22T17:50:45.510ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/system/networking/bgp-announce-set/{name_or_id}
19392024-08-22T17:50:45.510ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/networking/bgp-announce-set/{name_or_id}/announcement
19402024-08-22T17:50:45.510ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/networking/bgp-exported
19412024-08-22T17:50:45.510ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/networking/bgp-message-history
19422024-08-22T17:50:45.510ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/networking/bgp-routes-ipv4
19432024-08-22T17:50:45.510ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/networking/bgp-status
19442024-08-22T17:50:45.510ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/networking/loopback-address
19452024-08-22T17:50:45.510ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/system/networking/loopback-address
19462024-08-22T17:50:45.510ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/system/networking/loopback-address/{rack_id}/{switch_location}/{address}/{subnet_mask}
19472024-08-22T17:50:45.510ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/system/networking/switch-port-settings
19482024-08-22T17:50:45.510ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/networking/switch-port-settings
19492024-08-22T17:50:45.510ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/system/networking/switch-port-settings
19502024-08-22T17:50:45.510ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/networking/switch-port-settings/{port}
19512024-08-22T17:50:45.510ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/policy
19522024-08-22T17:50:45.510ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = PUT path = /v1/system/policy
19532024-08-22T17:50:45.510ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/roles
19542024-08-22T17:50:45.510ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/roles/{role_name}
19552024-08-22T17:50:45.510ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/silo-quotas
19562024-08-22T17:50:45.510ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/silos
19572024-08-22T17:50:45.510ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/system/silos
19582024-08-22T17:50:45.510ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/system/silos/{silo}
19592024-08-22T17:50:45.510ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/silos/{silo}
19602024-08-22T17:50:45.510ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/silos/{silo}/ip-pools
19612024-08-22T17:50:45.510ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/silos/{silo}/policy
19622024-08-22T17:50:45.510ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = PUT path = /v1/system/silos/{silo}/policy
19632024-08-22T17:50:45.510ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/silos/{silo}/quotas
19642024-08-22T17:50:45.510ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = PUT path = /v1/system/silos/{silo}/quotas
19652024-08-22T17:50:45.510ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = PUT path = /v1/system/update/repository
19662024-08-22T17:50:45.510ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/update/repository/{system_version}
19672024-08-22T17:50:45.510ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/users
19682024-08-22T17:50:45.510ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/users/{user_id}
19692024-08-22T17:50:45.510ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/users-builtin
19702024-08-22T17:50:45.510ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/users-builtin/{user}
19712024-08-22T17:50:45.510ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/utilization/silos
19722024-08-22T17:50:45.510ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/utilization/silos/{silo}
19732024-08-22T17:50:45.511ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/timeseries/query
19742024-08-22T17:50:45.511ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/timeseries/schema
19752024-08-22T17:50:45.511ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/users
19762024-08-22T17:50:45.511ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/utilization
19772024-08-22T17:50:45.511ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/vpc-firewall-rules
19782024-08-22T17:50:45.511ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = PUT path = /v1/vpc-firewall-rules
19792024-08-22T17:50:45.511ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/vpc-router-routes
19802024-08-22T17:50:45.511ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/vpc-router-routes
19812024-08-22T17:50:45.511ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/vpc-router-routes/{route}
19822024-08-22T17:50:45.511ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/vpc-router-routes/{route}
19832024-08-22T17:50:45.511ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = PUT path = /v1/vpc-router-routes/{route}
19842024-08-22T17:50:45.511ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/vpc-routers
19852024-08-22T17:50:45.511ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/vpc-routers
19862024-08-22T17:50:45.511ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/vpc-routers/{router}
19872024-08-22T17:50:45.511ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/vpc-routers/{router}
19882024-08-22T17:50:45.511ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = PUT path = /v1/vpc-routers/{router}
19892024-08-22T17:50:45.511ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/vpc-subnets
19902024-08-22T17:50:45.511ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/vpc-subnets
19912024-08-22T17:50:45.511ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/vpc-subnets/{subnet}
19922024-08-22T17:50:45.511ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/vpc-subnets/{subnet}
19932024-08-22T17:50:45.511ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = PUT path = /v1/vpc-subnets/{subnet}
19942024-08-22T17:50:45.511ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/vpc-subnets/{subnet}/network-interfaces
19952024-08-22T17:50:45.511ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/vpcs
19962024-08-22T17:50:45.511ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/vpcs
19972024-08-22T17:50:45.511ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/vpcs/{vpc}
19982024-08-22T17:50:45.511ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/vpcs/{vpc}
19992024-08-22T17:50:45.511ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = PUT path = /v1/vpcs/{vpc}
20002024-08-22T17:50:45.511ZINFOnexus (dropshot_external_techport): listening file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:205 local_addr = [fd00:1122:3344:101::c]:12228
20012024-08-22T17:50:45.511ZDEBGnexus (dropshot_external_techport): successfully registered DTrace USDT probes local_addr = [fd00:1122:3344:101::c]:12228
20022024-08-22T17:50:45.519ZDEBGnexus (nexus-producer-server): registered DTrace probes
20032024-08-22T17:50:45.524ZDEBGnexus (dropshot): registered endpoint local_addr = [fd00:1122:3344:101::c]:56994 method = GET path = /{producer_id}
20042024-08-22T17:50:45.524ZINFOnexus (dropshot): listening file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:205 local_addr = [fd00:1122:3344:101::c]:56994
20052024-08-22T17:50:45.524ZDEBGnexus (dropshot): successfully registered DTrace USDT probes local_addr = [fd00:1122:3344:101::c]:56994
20062024-08-22T17:50:45.524ZDEBGnexus (nexus-producer-server): Requested any available port, Dropshot server has been bound to [fd00:1122:3344:101::c]:56994
20072024-08-22T17:50:45.524ZDEBGnexus (nexus-producer-server): Nexus IP provided explicitly, will use it addr = [fd00:1122:3344:101::c]:12221
20082024-08-22T17:50:45.524ZDEBGnexus (nexus-producer-server): starting producer registration task
20092024-08-22T17:50:45.526ZINFOnexus (nexus-producer-server): starting oximeter metric producer server address = [fd00:1122:3344:101::c]:56994 file = oximeter/producer/src/lib.rs:283 interval = 10s producer_id = eaa260db-2b19-4ae1-97e8-5735c9f5d955
20102024-08-22T17:50:45.526ZDEBGnexus (producer-registration-task): registering / renewing oximeter producer lease with Nexus
20112024-08-22T17:50:45.526ZDEBGnexus (producer-registration-task): using nexus address for registration addr = [fd00:1122:3344:101::c]:12221
20122024-08-22T17:50:45.558ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
20132024-08-22T17:50:45.605ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
20142024-08-22T17:50:45.685ZDEBGnexus (producer-registration-task): client request body = Some(Body) method = POST uri = http://[fd00:1122:3344:101::c]:12221/metrics/producers
20152024-08-22T17:50:45.685ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::c]:34912
20162024-08-22T17:50:45.709ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:34912 req_id = 93393262-a081-4dc2-9204-dda548260e2e roles = RoleSet { roles: {} } uri = /metrics/producers
20172024-08-22T17:50:45.710ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:34912 req_id = 93393262-a081-4dc2-9204-dda548260e2e resource = Database result = Ok(()) uri = /metrics/producers
20182024-08-22T17:50:45.714ZDEBGnexus (BackgroundTasks): client request SledAgent = c961ede4-7532-4ca1-95a5-690b4d7c9b5a background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
20192024-08-22T17:50:45.714ZDEBGnexus (BackgroundTasks): client response SledAgent = c961ede4-7532-4ca1-95a5-690b4d7c9b5a background_task = vpc_route_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/vpc-routes", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "09983816-59cc-4218-b5f3-ebdc44743825", "content-length": "372", "date": "Thu, 22 Aug 2024 17:50:45 GMT"} })
20202024-08-22T17:50:45.714ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
20212024-08-22T17:50:45.715ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
20222024-08-22T17:50:45.717ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
20232024-08-22T17:50:45.718ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
20242024-08-22T17:50:45.720ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
20252024-08-22T17:50:45.723ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
20262024-08-22T17:50:45.723ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
20272024-08-22T17:50:45.726ZDEBGnexus (BackgroundTasks): client request SledAgent = c961ede4-7532-4ca1-95a5-690b4d7c9b5a background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
20282024-08-22T17:50:45.726ZDEBGnexus (BackgroundTasks): client response SledAgent = c961ede4-7532-4ca1-95a5-690b4d7c9b5a background_task = vpc_route_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/vpc-routes", query: None, fragment: None }, status: 204, headers: {"x-request-id": "647edbe8-e392-4298-83b0-1c1d67884632", "date": "Thu, 22 Aug 2024 17:50:45 GMT"} })
20292024-08-22T17:50:45.727ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 659.958849ms iteration = 3 status = {}
20302024-08-22T17:50:45.796ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
20312024-08-22T17:50:45.796ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::8)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "39cfc3a1-2037-40b9-9e49-6d21852b70fd", "date": "Thu, 22 Aug 2024 17:50:45 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
20322024-08-22T17:50:45.796ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
20332024-08-22T17:50:45.797ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::9)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "37d9d827-307f-4402-a8cb-d7eb32eff38a", "date": "Thu, 22 Aug 2024 17:50:45 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
20342024-08-22T17:50:45.797ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
20352024-08-22T17:50:45.799ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
20362024-08-22T17:50:45.799ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 388.815679ms iteration = 4 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
20372024-08-22T17:50:45.805ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
20382024-08-22T17:50:45.806ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
20392024-08-22T17:50:45.827ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
20402024-08-22T17:50:45.827ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:3::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "785def65-e413-4114-bd91-1b4b86955ecd", "date": "Thu, 22 Aug 2024 17:50:45 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
20412024-08-22T17:50:45.827ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
20422024-08-22T17:50:45.829ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:1::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "81829e43-2f58-4002-acd6-0c1c9cb30594", "date": "Thu, 22 Aug 2024 17:50:45 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
20432024-08-22T17:50:45.830ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
20442024-08-22T17:50:45.831ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:2::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "6d16d642-a269-40b9-a175-90219d1d6d7b", "date": "Thu, 22 Aug 2024 17:50:45 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
20452024-08-22T17:50:45.831ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
20462024-08-22T17:50:45.833ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
20472024-08-22T17:50:45.833ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 417.591248ms iteration = 4 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
20482024-08-22T17:50:45.837ZINFOnexus: registered oximeter collector client file = nexus/src/app/oximeter.rs:338 id = 7fc6f696-ffb5-46eb-ae34-da7098d2b35a
20492024-08-22T17:50:45.837ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:34912 req_id = 93393262-a081-4dc2-9204-dda548260e2e roles = RoleSet { roles: {} } uri = /metrics/producers
20502024-08-22T17:50:45.837ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:34912 req_id = 93393262-a081-4dc2-9204-dda548260e2e resource = Database result = Ok(()) uri = /metrics/producers
20512024-08-22T17:50:45.839ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:147 port = SwitchPort { id: c772d372-7448-4ce0-b8d1-efa645d18ed1, rack_id: c66d93a6-e2c3-4b19-9564-3b58bad50561, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(9ff05247-84aa-47d8-8619-ce0a8dbbaf7c) } rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 switch_location = Switch0
20522024-08-22T17:50:45.839ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
20532024-08-22T17:50:45.839ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
20542024-08-22T17:50:45.869ZDEBGnexus: client request body = Some(Body) method = POST oximeter-collector = 7fc6f696-ffb5-46eb-ae34-da7098d2b35a uri = http://[fd00:1122:3344:101::d]:12223/producers
20552024-08-22T17:50:45.870ZDEBGnexus: client response oximeter-collector = 7fc6f696-ffb5-46eb-ae34-da7098d2b35a result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::d)), port: Some(12223), path: "/producers", query: None, fragment: None }, status: 204, headers: {"x-request-id": "9e40ea25-a9ba-4c81-ac65-e968b91b4265", "date": "Thu, 22 Aug 2024 17:50:45 GMT"} })
20562024-08-22T17:50:45.870ZINFOnexus: assigned collector to new producer collector_id = 7fc6f696-ffb5-46eb-ae34-da7098d2b35a file = nexus/src/app/oximeter.rs:125 producer_id = eaa260db-2b19-4ae1-97e8-5735c9f5d955
20572024-08-22T17:50:45.878ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 184531 local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:34912 req_id = 93393262-a081-4dc2-9204-dda548260e2e response_code = 201 uri = /metrics/producers
20582024-08-22T17:50:45.880ZDEBGnexus (producer-registration-task): client response result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::c)), port: Some(12221), path: "/metrics/producers", query: None, fragment: None }, status: 201, headers: {"content-type": "application/json", "x-request-id": "93393262-a081-4dc2-9204-dda548260e2e", "content-length": "41", "date": "Thu, 22 Aug 2024 17:50:45 GMT"} })
20592024-08-22T17:50:45.882ZDEBGnexus (producer-registration-task): registered with nexus successfully
20602024-08-22T17:50:45.882ZDEBGnexus (producer-registration-task): pausing until time to renew lease lease_duration = 600s wait_period = 150s
20612024-08-22T17:50:46.046ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
20622024-08-22T17:50:46.047ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/port/qsfp0/settings", query: Some("tag=omicron"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "1edc80f0-47bb-4797-b1d3-bb04c6e6b8fe", "content-length": "121", "date": "Thu, 22 Aug 2024 17:50:45 GMT"} })
20632024-08-22T17:50:46.047ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1714 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G } }} } switch_port_id = "qsfp0"
20642024-08-22T17:50:46.047ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1741 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 9ff05247-84aa-47d8-8619-ce0a8dbbaf7c
20652024-08-22T17:50:46.047ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:379 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
20662024-08-22T17:50:46.047ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
20672024-08-22T17:50:46.047ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:392 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 routes = {}
20682024-08-22T17:50:46.047ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:398 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 routes = {}
20692024-08-22T17:50:46.047ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:418 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561
20702024-08-22T17:50:46.047ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
20712024-08-22T17:50:46.047ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
20722024-08-22T17:50:46.156ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
20732024-08-22T17:50:46.156ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv4", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "be65989d-d464-4054-8d75-d39d2f342af5", "content-length": "2", "date": "Thu, 22 Aug 2024 17:50:45 GMT"} })
20742024-08-22T17:50:46.156ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
20752024-08-22T17:50:46.157ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "fac3b430-c542-4903-bec1-741d73b2e6d3", "content-length": "52", "date": "Thu, 22 Aug 2024 17:50:45 GMT"} })
20762024-08-22T17:50:46.157ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0" }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:467 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 switch_location = Switch0
20772024-08-22T17:50:46.157ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
20782024-08-22T17:50:46.305ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/switch-ports", query: None, fragment: None }, status: 204, headers: {"x-request-id": "a7ab3986-9db5-4d3f-8906-2e21aae4241b", "date": "Thu, 22 Aug 2024 17:50:46 GMT"} })
20792024-08-22T17:50:46.305ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:770 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 switch = Switch0
20802024-08-22T17:50:46.305ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
20812024-08-22T17:50:46.305ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/network-bootstore-config", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "0f2ec28e-1534-4e55-8687-cce029106d27", "content-length": "667", "date": "Thu, 22 Aug 2024 17:50:46 GMT"} })
20822024-08-22T17:50:46.305ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
20832024-08-22T17:50:46.306ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
20842024-08-22T17:50:46.385ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
20852024-08-22T17:50:46.385ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
20862024-08-22T17:50:46.399ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
20872024-08-22T17:50:46.399ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
20882024-08-22T17:50:46.400ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
20892024-08-22T17:50:46.401ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
20902024-08-22T17:50:46.407ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 1.339971084s iteration = 3 status = {}
20912024-08-22T17:50:55.067ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 8 reason = Timeout
20922024-08-22T17:50:55.068ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
20932024-08-22T17:50:55.068ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 8 reason = Timeout
20942024-08-22T17:50:55.068ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
20952024-08-22T17:50:55.068ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
20962024-08-22T17:50:55.068ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
20972024-08-22T17:50:55.070ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20982024-08-22T17:50:55.070ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
20992024-08-22T17:50:55.070ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
21002024-08-22T17:50:55.072ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
21012024-08-22T17:50:55.072ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
21022024-08-22T17:50:55.072ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
21032024-08-22T17:50:55.074ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
21042024-08-22T17:50:55.074ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
21052024-08-22T17:50:55.076ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21062024-08-22T17:50:55.078ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 147bdc10-344c-4ee8-a219-603cfc66e6d2, lookup_type: ById(147bdc10-344c-4ee8-a219-603cfc66e6d2) } result = Ok(())
21072024-08-22T17:50:55.078ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
21082024-08-22T17:50:55.078ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
21092024-08-22T17:50:55.085ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 147bdc10-344c-4ee8-a219-603cfc66e6d2 original_time_created = 2024-08-22 17:49:18.445237 UTC target_id = 147bdc10-344c-4ee8-a219-603cfc66e6d2 time_created = 2024-08-22 17:49:18.445237 UTC
21102024-08-22T17:50:55.085ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 17.959603ms iteration = 8 status = {"status":"target blueprint unchanged","target_id":"147bdc10-344c-4ee8-a219-603cfc66e6d2","time_created":"2024-08-22T17:49:18.445237Z"}
21112024-08-22T17:50:55.096ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 28.93618ms iteration = 8 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
21122024-08-22T17:50:55.910ZINFOnexus (dropshot): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::c]:56994 remote_addr = [fd00:1122:3344:101::d]:40670
21132024-08-22T17:50:55.916ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 6059 local_addr = [fd00:1122:3344:101::c]:56994 method = GET remote_addr = [fd00:1122:3344:101::d]:40670 req_id = 6f880aaf-79cc-4591-b7aa-65a78b930474 response_code = 200 uri = /eaa260db-2b19-4ae1-97e8-5735c9f5d955
21142024-08-22T17:51:05.068ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 9 reason = Timeout
21152024-08-22T17:51:05.068ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 9 reason = Timeout
21162024-08-22T17:51:05.068ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
21172024-08-22T17:51:05.068ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
21182024-08-22T17:51:05.068ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
21192024-08-22T17:51:05.068ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
21202024-08-22T17:51:05.070ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21212024-08-22T17:51:05.070ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
21222024-08-22T17:51:05.071ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
21232024-08-22T17:51:05.072ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
21242024-08-22T17:51:05.072ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
21252024-08-22T17:51:05.072ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
21262024-08-22T17:51:05.077ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
21272024-08-22T17:51:05.077ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
21282024-08-22T17:51:05.078ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21292024-08-22T17:51:05.081ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 147bdc10-344c-4ee8-a219-603cfc66e6d2, lookup_type: ById(147bdc10-344c-4ee8-a219-603cfc66e6d2) } result = Ok(())
21302024-08-22T17:51:05.081ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
21312024-08-22T17:51:05.081ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
21322024-08-22T17:51:05.129ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 147bdc10-344c-4ee8-a219-603cfc66e6d2 original_time_created = 2024-08-22 17:49:18.445237 UTC target_id = 147bdc10-344c-4ee8-a219-603cfc66e6d2 time_created = 2024-08-22 17:49:18.445237 UTC
21332024-08-22T17:51:05.129ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 61.084806ms iteration = 9 status = {"status":"target blueprint unchanged","target_id":"147bdc10-344c-4ee8-a219-603cfc66e6d2","time_created":"2024-08-22T17:49:18.445237Z"}
21342024-08-22T17:51:05.193ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 124.811365ms iteration = 9 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
21352024-08-22T17:51:05.911ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 279 local_addr = [fd00:1122:3344:101::c]:56994 method = GET remote_addr = [fd00:1122:3344:101::d]:40670 req_id = f5285b40-bc27-49fe-9aaa-989a306b2e41 response_code = 200 uri = /eaa260db-2b19-4ae1-97e8-5735c9f5d955
21362024-08-22T17:51:15.068ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 10 reason = Timeout
21372024-08-22T17:51:15.068ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 4 reason = Timeout
21382024-08-22T17:51:15.068ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
21392024-08-22T17:51:15.068ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
21402024-08-22T17:51:15.068ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 4 reason = Timeout
21412024-08-22T17:51:15.068ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 4 reason = Timeout
21422024-08-22T17:51:15.068ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
21432024-08-22T17:51:15.068ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
21442024-08-22T17:51:15.068ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 4 reason = Timeout
21452024-08-22T17:51:15.068ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
21462024-08-22T17:51:15.068ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 4 reason = Timeout
21472024-08-22T17:51:15.068ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
21482024-08-22T17:51:15.068ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 4 reason = Timeout
21492024-08-22T17:51:15.068ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 4 reason = Timeout
21502024-08-22T17:51:15.068ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 10 reason = Timeout
21512024-08-22T17:51:15.068ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
21522024-08-22T17:51:15.068ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
21532024-08-22T17:51:15.068ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 5 reason = Timeout
21542024-08-22T17:51:15.068ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
21552024-08-22T17:51:15.068ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 4 reason = Timeout
21562024-08-22T17:51:15.068ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 4 reason = Timeout
21572024-08-22T17:51:15.068ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
21582024-08-22T17:51:15.068ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 4 reason = Timeout
21592024-08-22T17:51:15.068ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
21602024-08-22T17:51:15.068ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
21612024-08-22T17:51:15.068ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
21622024-08-22T17:51:15.068ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 4 reason = Timeout
21632024-08-22T17:51:15.068ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
21642024-08-22T17:51:15.068ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
21652024-08-22T17:51:15.068ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
21662024-08-22T17:51:15.068ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
21672024-08-22T17:51:15.068ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
21682024-08-22T17:51:15.068ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
21692024-08-22T17:51:15.069ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
21702024-08-22T17:51:15.069ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
21712024-08-22T17:51:15.069ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
21722024-08-22T17:51:15.069ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
21732024-08-22T17:51:15.069ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
21742024-08-22T17:51:15.069ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
21752024-08-22T17:51:15.069ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
21762024-08-22T17:51:15.069ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
21772024-08-22T17:51:15.070ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 4 reason = Timeout
21782024-08-22T17:51:15.070ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
21792024-08-22T17:51:15.070ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
21802024-08-22T17:51:15.071ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21812024-08-22T17:51:15.072ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-c961ede4-7532-4ca1-95a5-690b4d7c9b5a.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-c961ede4-7532-4ca1-95a5-690b4d7c9b5a.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1663, tv_nsec: 605815945 } })
21822024-08-22T17:51:15.073ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1050
21832024-08-22T17:51:15.074ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21842024-08-22T17:51:15.074ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
21852024-08-22T17:51:15.074ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
21862024-08-22T17:51:15.074ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 6.284222ms iteration = 4 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
21872024-08-22T17:51:15.075ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
21882024-08-22T17:51:15.076ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
21892024-08-22T17:51:15.076ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
21902024-08-22T17:51:15.076ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
21912024-08-22T17:51:15.079ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21922024-08-22T17:51:15.080ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21932024-08-22T17:51:15.081ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-c961ede4-7532-4ca1-95a5-690b4d7c9b5a.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-c961ede4-7532-4ca1-95a5-690b4d7c9b5a.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1663, tv_nsec: 614662791 } })
21942024-08-22T17:51:15.082ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1050
21952024-08-22T17:51:15.082ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
21962024-08-22T17:51:15.082ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
21972024-08-22T17:51:15.082ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
21982024-08-22T17:51:15.082ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
21992024-08-22T17:51:15.082ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
22002024-08-22T17:51:15.083ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
22012024-08-22T17:51:15.084ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
22022024-08-22T17:51:15.084ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
22032024-08-22T17:51:15.085ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22042024-08-22T17:51:15.086ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
22052024-08-22T17:51:15.086ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
22062024-08-22T17:51:15.089ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 147bdc10-344c-4ee8-a219-603cfc66e6d2, lookup_type: ById(147bdc10-344c-4ee8-a219-603cfc66e6d2) } result = Ok(())
22072024-08-22T17:51:15.089ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
22082024-08-22T17:51:15.089ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
22092024-08-22T17:51:15.091ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22102024-08-22T17:51:15.093ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
22112024-08-22T17:51:15.093ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
22122024-08-22T17:51:15.094ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
22132024-08-22T17:51:15.107ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
22142024-08-22T17:51:15.108ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
22152024-08-22T17:51:15.109ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 41.174373ms iteration = 10 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
22162024-08-22T17:51:15.112ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22172024-08-22T17:51:15.114ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
22182024-08-22T17:51:15.114ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
22192024-08-22T17:51:15.115ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
22202024-08-22T17:51:15.115ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
22212024-08-22T17:51:15.115ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
22222024-08-22T17:51:15.115ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
22232024-08-22T17:51:15.116ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
22242024-08-22T17:51:15.116ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 48.013879ms iteration = 4 status = {"region_replacement_started_err":0,"region_replacement_started_ok":0}
22252024-08-22T17:51:15.117ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
22262024-08-22T17:51:15.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
22272024-08-22T17:51:15.121ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:55
22282024-08-22T17:51:15.121ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
22292024-08-22T17:51:15.121ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
22302024-08-22T17:51:15.122ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
22312024-08-22T17:51:15.124ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22322024-08-22T17:51:15.126ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
22332024-08-22T17:51:15.126ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
22342024-08-22T17:51:15.126ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
22352024-08-22T17:51:15.128ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1058 zone_address = fd00:1122:3344:101::2
22362024-08-22T17:51:15.128ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
22372024-08-22T17:51:15.128ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "89b7c786-08bd-4750-8f46-09d111277e79", "content-length": "26", "date": "Thu, 22 Aug 2024 17:51:14 GMT"} })
22382024-08-22T17:51:15.128ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1063 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
22392024-08-22T17:51:15.128ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1058 zone_address = fd00:1122:3344:101::2
22402024-08-22T17:51:15.128ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
22412024-08-22T17:51:15.128ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "326d06a7-b111-4de6-bd7a-b1419808b50b", "content-length": "26", "date": "Thu, 22 Aug 2024 17:51:14 GMT"} })
22422024-08-22T17:51:15.128ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1063 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
22432024-08-22T17:51:15.129ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1095 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
22442024-08-22T17:51:15.130ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1095 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
22452024-08-22T17:51:15.131ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 60.767345ms iteration = 4 status = {"errors":[],"garbage_collect_requested":[]}
22462024-08-22T17:51:15.145ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
22472024-08-22T17:51:15.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
22482024-08-22T17:51:15.148ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
22492024-08-22T17:51:15.149ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
22502024-08-22T17:51:15.150ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22512024-08-22T17:51:15.152ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: c961ede4-7532-4ca1-95a5-690b4d7c9b5a, lookup_type: ById(c961ede4-7532-4ca1-95a5-690b4d7c9b5a) } result = Ok(())
22522024-08-22T17:51:15.152ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
22532024-08-22T17:51:15.153ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
22542024-08-22T17:51:15.154ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 86.075695ms iteration = 4 status = 0
22552024-08-22T17:51:15.155ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 86.880209ms iteration = 4 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
22562024-08-22T17:51:15.161ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:454 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
22572024-08-22T17:51:15.161ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 92.720169ms iteration = 4 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
22582024-08-22T17:51:15.184ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
22592024-08-22T17:51:15.184ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
22602024-08-22T17:51:15.190ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 121.652818ms iteration = 5 status = {"physical_disks_added":0}
22612024-08-22T17:51:15.192ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
22622024-08-22T17:51:15.192ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
22632024-08-22T17:51:15.193ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
22642024-08-22T17:51:15.193ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "3fe5bff9-93b6-4409-8f90-49e863599c69", "content-length": "1", "date": "Thu, 22 Aug 2024 17:51:14 GMT"} })
22652024-08-22T17:51:15.194ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22662024-08-22T17:51:15.195ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
22672024-08-22T17:51:15.195ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
22682024-08-22T17:51:15.196ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
22692024-08-22T17:51:15.196ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
22702024-08-22T17:51:15.196ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
22712024-08-22T17:51:15.199ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 131.374669ms iteration = 4 status = {}
22722024-08-22T17:51:15.200ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 147bdc10-344c-4ee8-a219-603cfc66e6d2 original_time_created = 2024-08-22 17:49:18.445237 UTC target_id = 147bdc10-344c-4ee8-a219-603cfc66e6d2 time_created = 2024-08-22 17:49:18.445237 UTC
22732024-08-22T17:51:15.200ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 132.477925ms iteration = 10 status = {"status":"target blueprint unchanged","target_id":"147bdc10-344c-4ee8-a219-603cfc66e6d2","time_created":"2024-08-22T17:49:18.445237Z"}
22742024-08-22T17:51:15.209ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22752024-08-22T17:51:15.211ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
22762024-08-22T17:51:15.211ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
22772024-08-22T17:51:15.211ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
22782024-08-22T17:51:15.226ZDEBGnexus (BackgroundTasks): client request SledAgent = c961ede4-7532-4ca1-95a5-690b4d7c9b5a background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
22792024-08-22T17:51:15.226ZDEBGnexus (BackgroundTasks): client response SledAgent = c961ede4-7532-4ca1-95a5-690b4d7c9b5a background_task = vpc_route_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/vpc-routes", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "9fab3b32-722f-4a87-aa63-ebb6401a7392", "content-length": "372", "date": "Thu, 22 Aug 2024 17:51:15 GMT"} })
22802024-08-22T17:51:15.226ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
22812024-08-22T17:51:15.227ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
22822024-08-22T17:51:15.228ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22832024-08-22T17:51:15.228ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 160.096987ms iteration = 4 status = null
22842024-08-22T17:51:15.229ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
22852024-08-22T17:51:15.229ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
22862024-08-22T17:51:15.229ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
22872024-08-22T17:51:15.230ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
22882024-08-22T17:51:15.230ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
22892024-08-22T17:51:15.230ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-c961ede4-7532-4ca1-95a5-690b4d7c9b5a.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-c961ede4-7532-4ca1-95a5-690b4d7c9b5a.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1663, tv_nsec: 763776111 } })
22902024-08-22T17:51:15.231ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1050 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561
22912024-08-22T17:51:15.232ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
22922024-08-22T17:51:15.232ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
22932024-08-22T17:51:15.234ZDEBGnexus (BackgroundTasks): client request SledAgent = c961ede4-7532-4ca1-95a5-690b4d7c9b5a background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
22942024-08-22T17:51:15.235ZDEBGnexus (BackgroundTasks): client response SledAgent = c961ede4-7532-4ca1-95a5-690b4d7c9b5a background_task = vpc_route_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/vpc-routes", query: None, fragment: None }, status: 204, headers: {"x-request-id": "0bcede61-b44d-47a4-aaaf-84af9bfd93a6", "date": "Thu, 22 Aug 2024 17:51:15 GMT"} })
22952024-08-22T17:51:15.236ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 168.43025ms iteration = 4 status = {}
22962024-08-22T17:51:15.261ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1058 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 zone_address = fd00:1122:3344:101::2
22972024-08-22T17:51:15.261ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
22982024-08-22T17:51:15.262ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "7efc6895-a850-4e4f-81a7-65297a844d70", "content-length": "26", "date": "Thu, 22 Aug 2024 17:51:14 GMT"} })
22992024-08-22T17:51:15.262ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1063 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
23002024-08-22T17:51:15.263ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1095 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561
23012024-08-22T17:51:15.267ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:55
23022024-08-22T17:51:15.267ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:220 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
23032024-08-22T17:51:15.267ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 198.726527ms iteration = 4 status = {"destroyed_active_vmms":0,"error":null,"saga_start_failures":0,"sagas_completed":0,"sagas_failed":0,"sagas_started":0,"terminated_active_migrations":0}
23042024-08-22T17:51:15.334ZDEBGnexus (BackgroundTasks): client request SledAgent = c961ede4-7532-4ca1-95a5-690b4d7c9b5a background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
23052024-08-22T17:51:15.334ZDEBGnexus (BackgroundTasks): client response SledAgent = c961ede4-7532-4ca1-95a5-690b4d7c9b5a background_task = v2p_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/v2p", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "4347b500-035f-41bc-94cb-a57dca0f8aea", "content-length": "691", "date": "Thu, 22 Aug 2024 17:51:15 GMT"} })
23062024-08-22T17:51:15.334ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = buskin
23072024-08-22T17:51:15.334ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = buskin
23082024-08-22T17:51:15.334ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = buskin
23092024-08-22T17:51:15.335ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 266.752619ms iteration = 4 status = {}
23102024-08-22T17:51:15.383ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
23112024-08-22T17:51:15.383ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
23122024-08-22T17:51:15.384ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:147 port = SwitchPort { id: c772d372-7448-4ce0-b8d1-efa645d18ed1, rack_id: c66d93a6-e2c3-4b19-9564-3b58bad50561, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(9ff05247-84aa-47d8-8619-ce0a8dbbaf7c) } rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 switch_location = Switch0
23132024-08-22T17:51:15.385ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
23142024-08-22T17:51:15.385ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
23152024-08-22T17:51:15.394ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
23162024-08-22T17:51:15.394ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/port/qsfp0/settings", query: Some("tag=omicron"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "ac6bf4bf-7e9c-4e59-b88c-782a573112e0", "content-length": "121", "date": "Thu, 22 Aug 2024 17:51:15 GMT"} })
23172024-08-22T17:51:15.394ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1714 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G } }} } switch_port_id = "qsfp0"
23182024-08-22T17:51:15.394ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1741 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 9ff05247-84aa-47d8-8619-ce0a8dbbaf7c
23192024-08-22T17:51:15.394ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:379 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
23202024-08-22T17:51:15.394ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
23212024-08-22T17:51:15.394ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:392 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 routes = {}
23222024-08-22T17:51:15.394ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:398 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 routes = {}
23232024-08-22T17:51:15.394ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:418 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561
23242024-08-22T17:51:15.395ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
23252024-08-22T17:51:15.395ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
23262024-08-22T17:51:15.396ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
23272024-08-22T17:51:15.396ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv4", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "77e8fb2a-8c32-4247-a306-28ad41e3fb1e", "content-length": "2", "date": "Thu, 22 Aug 2024 17:51:15 GMT"} })
23282024-08-22T17:51:15.396ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
23292024-08-22T17:51:15.397ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "d130678b-e737-4b63-99a9-1db81e490ccf", "content-length": "52", "date": "Thu, 22 Aug 2024 17:51:15 GMT"} })
23302024-08-22T17:51:15.397ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0" }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:467 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 switch_location = Switch0
23312024-08-22T17:51:15.397ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
23322024-08-22T17:51:15.500ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/switch-ports", query: None, fragment: None }, status: 204, headers: {"x-request-id": "f7ed9aa4-f7bf-4261-aed5-f24d4cfb315e", "date": "Thu, 22 Aug 2024 17:51:15 GMT"} })
23332024-08-22T17:51:15.500ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:770 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 switch = Switch0
23342024-08-22T17:51:15.500ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
23352024-08-22T17:51:15.616ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/network-bootstore-config", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "dd5b2132-fe7a-4ec0-a50f-08c831b4a4be", "content-length": "667", "date": "Thu, 22 Aug 2024 17:51:15 GMT"} })
23362024-08-22T17:51:15.616ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
23372024-08-22T17:51:15.617ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
23382024-08-22T17:51:15.720ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
23392024-08-22T17:51:15.720ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
23402024-08-22T17:51:15.723ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
23412024-08-22T17:51:15.723ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
23422024-08-22T17:51:15.820ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
23432024-08-22T17:51:15.820ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
23442024-08-22T17:51:15.827ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 758.768271ms iteration = 4 status = {}
23452024-08-22T17:51:15.911ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 199 local_addr = [fd00:1122:3344:101::c]:56994 method = GET remote_addr = [fd00:1122:3344:101::d]:40670 req_id = 5da2785d-c624-4e36-9cd5-6451a41be562 response_code = 200 uri = /eaa260db-2b19-4ae1-97e8-5735c9f5d955
23462024-08-22T17:51:25.068ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 11 reason = Timeout
23472024-08-22T17:51:25.068ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 11 reason = Timeout
23482024-08-22T17:51:25.068ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
23492024-08-22T17:51:25.068ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
23502024-08-22T17:51:25.068ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
23512024-08-22T17:51:25.068ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
23522024-08-22T17:51:25.070ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23532024-08-22T17:51:25.073ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
23542024-08-22T17:51:25.073ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
23552024-08-22T17:51:25.073ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
23562024-08-22T17:51:25.075ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
23572024-08-22T17:51:25.075ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
23582024-08-22T17:51:25.077ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23592024-08-22T17:51:25.079ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 147bdc10-344c-4ee8-a219-603cfc66e6d2, lookup_type: ById(147bdc10-344c-4ee8-a219-603cfc66e6d2) } result = Ok(())
23602024-08-22T17:51:25.079ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
23612024-08-22T17:51:25.080ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
23622024-08-22T17:51:25.103ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
23632024-08-22T17:51:25.103ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
23642024-08-22T17:51:25.104ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 36.165207ms iteration = 11 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
23652024-08-22T17:51:25.202ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 147bdc10-344c-4ee8-a219-603cfc66e6d2 original_time_created = 2024-08-22 17:49:18.445237 UTC target_id = 147bdc10-344c-4ee8-a219-603cfc66e6d2 time_created = 2024-08-22 17:49:18.445237 UTC
23662024-08-22T17:51:25.202ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 134.340174ms iteration = 11 status = {"status":"target blueprint unchanged","target_id":"147bdc10-344c-4ee8-a219-603cfc66e6d2","time_created":"2024-08-22T17:49:18.445237Z"}
23672024-08-22T17:51:25.912ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 894 local_addr = [fd00:1122:3344:101::c]:56994 method = GET remote_addr = [fd00:1122:3344:101::d]:40670 req_id = 993db054-b065-43cb-8ca6-8b829e1960b8 response_code = 200 uri = /eaa260db-2b19-4ae1-97e8-5735c9f5d955
23682024-08-22T17:51:35.068ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 12 reason = Timeout
23692024-08-22T17:51:35.068ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 12 reason = Timeout
23702024-08-22T17:51:35.068ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
23712024-08-22T17:51:35.068ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
23722024-08-22T17:51:35.068ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
23732024-08-22T17:51:35.068ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
23742024-08-22T17:51:35.070ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
23752024-08-22T17:51:35.070ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
23762024-08-22T17:51:35.071ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23772024-08-22T17:51:35.072ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.987291ms iteration = 12 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
23782024-08-22T17:51:35.073ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
23792024-08-22T17:51:35.073ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
23802024-08-22T17:51:35.073ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
23812024-08-22T17:51:35.075ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
23822024-08-22T17:51:35.075ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
23832024-08-22T17:51:35.077ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23842024-08-22T17:51:35.079ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 147bdc10-344c-4ee8-a219-603cfc66e6d2, lookup_type: ById(147bdc10-344c-4ee8-a219-603cfc66e6d2) } result = Ok(())
23852024-08-22T17:51:35.079ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
23862024-08-22T17:51:35.079ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
23872024-08-22T17:51:35.087ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 147bdc10-344c-4ee8-a219-603cfc66e6d2 original_time_created = 2024-08-22 17:49:18.445237 UTC target_id = 147bdc10-344c-4ee8-a219-603cfc66e6d2 time_created = 2024-08-22 17:49:18.445237 UTC
23882024-08-22T17:51:35.087ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 19.133029ms iteration = 12 status = {"status":"target blueprint unchanged","target_id":"147bdc10-344c-4ee8-a219-603cfc66e6d2","time_created":"2024-08-22T17:49:18.445237Z"}
23892024-08-22T17:51:35.911ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 227 local_addr = [fd00:1122:3344:101::c]:56994 method = GET remote_addr = [fd00:1122:3344:101::d]:40670 req_id = 5d7a5a8a-fa98-4a8c-b0fd-e4226b9cca0d response_code = 200 uri = /eaa260db-2b19-4ae1-97e8-5735c9f5d955
23902024-08-22T17:51:45.068ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 3 reason = Timeout
23912024-08-22T17:51:45.069ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 3 reason = Timeout
23922024-08-22T17:51:45.069ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 3 reason = Timeout
23932024-08-22T17:51:45.069ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
23942024-08-22T17:51:45.069ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2024-08-22 17:41:45.068963346 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
23952024-08-22T17:51:45.069ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
23962024-08-22T17:51:45.069ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 3 reason = Timeout
23972024-08-22T17:51:45.069ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
23982024-08-22T17:51:45.069ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 5 reason = Timeout
23992024-08-22T17:51:45.069ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
24002024-08-22T17:51:45.069ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 5 reason = Timeout
24012024-08-22T17:51:45.069ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 5 reason = Timeout
24022024-08-22T17:51:45.069ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 13 reason = Timeout
24032024-08-22T17:51:45.069ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 5 reason = Timeout
24042024-08-22T17:51:45.069ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
24052024-08-22T17:51:45.069ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 5 reason = Timeout
24062024-08-22T17:51:45.069ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
24072024-08-22T17:51:45.069ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
24082024-08-22T17:51:45.069ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 5 reason = Timeout
24092024-08-22T17:51:45.069ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 13 reason = Timeout
24102024-08-22T17:51:45.069ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
24112024-08-22T17:51:45.069ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 6 reason = Timeout
24122024-08-22T17:51:45.069ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
24132024-08-22T17:51:45.069ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
24142024-08-22T17:51:45.069ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 3 reason = Timeout
24152024-08-22T17:51:45.069ZINFOnexus (BackgroundTasks): Abandoned VMM reaper running background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:76
24162024-08-22T17:51:45.069ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 3 reason = Timeout
24172024-08-22T17:51:45.069ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
24182024-08-22T17:51:45.069ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
24192024-08-22T17:51:45.069ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 5 reason = Timeout
24202024-08-22T17:51:45.069ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
24212024-08-22T17:51:45.069ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 5 reason = Timeout
24222024-08-22T17:51:45.069ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
24232024-08-22T17:51:45.069ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 5 reason = Timeout
24242024-08-22T17:51:45.069ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
24252024-08-22T17:51:45.069ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 3 reason = Timeout
24262024-08-22T17:51:45.069ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:56
24272024-08-22T17:51:45.069ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
24282024-08-22T17:51:45.069ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 3 reason = Timeout
24292024-08-22T17:51:45.069ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
24302024-08-22T17:51:45.069ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
24312024-08-22T17:51:45.069ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 4 reason = Timeout
24322024-08-22T17:51:45.069ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:80 reason = blueprint disabled target_id = 147bdc10-344c-4ee8-a219-603cfc66e6d2
24332024-08-22T17:51:45.069ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 3 reason = Timeout
24342024-08-22T17:51:45.069ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 16.74\u{b5}s iteration = 4 status = {"error":"blueprint disabled","target_id":"147bdc10-344c-4ee8-a219-603cfc66e6d2"}
24352024-08-22T17:51:45.069ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 5 reason = Timeout
24362024-08-22T17:51:45.069ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 5 reason = Timeout
24372024-08-22T17:51:45.069ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
24382024-08-22T17:51:45.069ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
24392024-08-22T17:51:45.069ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 5 reason = Timeout
24402024-08-22T17:51:45.069ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
24412024-08-22T17:51:45.069ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 5 reason = Timeout
24422024-08-22T17:51:45.069ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
24432024-08-22T17:51:45.069ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
24442024-08-22T17:51:45.069ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
24452024-08-22T17:51:45.069ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
24462024-08-22T17:51:45.069ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
24472024-08-22T17:51:45.069ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
24482024-08-22T17:51:45.069ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
24492024-08-22T17:51:45.069ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
24502024-08-22T17:51:45.069ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
24512024-08-22T17:51:45.069ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
24522024-08-22T17:51:45.069ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
24532024-08-22T17:51:45.069ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
24542024-08-22T17:51:45.069ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
24552024-08-22T17:51:45.069ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
24562024-08-22T17:51:45.069ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
24572024-08-22T17:51:45.069ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
24582024-08-22T17:51:45.069ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
24592024-08-22T17:51:45.069ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
24602024-08-22T17:51:45.069ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
24612024-08-22T17:51:45.069ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
24622024-08-22T17:51:45.069ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
24632024-08-22T17:51:45.069ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
24642024-08-22T17:51:45.071ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 5 reason = Timeout
24652024-08-22T17:51:45.071ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
24662024-08-22T17:51:45.071ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
24672024-08-22T17:51:45.071ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("9488ce0a-6732-4f75-baf9-4ecfa406df91.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("afac2eab-cadf-48d0-94a7-ad562ade88c6.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("afac2eab-cadf-48d0-94a7-ad562ade88c6.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::9))) }], valid_until: Instant { tv_sec: 1693, tv_nsec: 603536476 } })
24682024-08-22T17:51:45.071ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
24692024-08-22T17:51:45.071ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
24702024-08-22T17:51:45.072ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("6507c79b-9a18-46c7-930d-c98e8662b87f.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("a09ea923-b85c-428d-b289-6936a0c2542c.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("c0ef020c-52c1-4b9a-a085-445fab31264f.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("c0ef020c-52c1-4b9a-a085-445fab31264f.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:3::1))) }], valid_until: Instant { tv_sec: 1693, tv_nsec: 604794783 } })
24712024-08-22T17:51:45.073ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 4.161471ms iteration = 5 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
24722024-08-22T17:51:45.073ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24732024-08-22T17:51:45.074ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2024-08-22 17:41:45.068963346 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
24742024-08-22T17:51:45.074ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 5.917071ms iteration = 3 status = {"expiration":"2024-08-22T17:41:45.068963346Z","pruned":[]}
24752024-08-22T17:51:45.075ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24762024-08-22T17:51:45.076ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
24772024-08-22T17:51:45.076ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
24782024-08-22T17:51:45.076ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
24792024-08-22T17:51:45.076ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
24802024-08-22T17:51:45.076ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 7.963821ms iteration = 3 status = {"addresses":["[fd00:1122:3344:101::8]:5353","[fd00:1122:3344:101::9]:5353"]}
24812024-08-22T17:51:45.077ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
24822024-08-22T17:51:45.078ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
24832024-08-22T17:51:45.078ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
24842024-08-22T17:51:45.078ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
24852024-08-22T17:51:45.078ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
24862024-08-22T17:51:45.078ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
24872024-08-22T17:51:45.078ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
24882024-08-22T17:51:45.079ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24892024-08-22T17:51:45.079ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } dns_group = internal
24902024-08-22T17:51:45.079ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 10.091302ms iteration = 3 status = {"addresses":["[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:3::1]:5353"]}
24912024-08-22T17:51:45.080ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
24922024-08-22T17:51:45.080ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
24932024-08-22T17:51:45.081ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
24942024-08-22T17:51:45.081ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24952024-08-22T17:51:45.081ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
24962024-08-22T17:51:45.081ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
24972024-08-22T17:51:45.081ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
24982024-08-22T17:51:45.081ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
24992024-08-22T17:51:45.082ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
25002024-08-22T17:51:45.082ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
25012024-08-22T17:51:45.082ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25022024-08-22T17:51:45.083ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
25032024-08-22T17:51:45.084ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25042024-08-22T17:51:45.084ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
25052024-08-22T17:51:45.084ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
25062024-08-22T17:51:45.085ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
25072024-08-22T17:51:45.085ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-c961ede4-7532-4ca1-95a5-690b4d7c9b5a.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-c961ede4-7532-4ca1-95a5-690b4d7c9b5a.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1693, tv_nsec: 617993711 } })
25082024-08-22T17:51:45.086ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1050
25092024-08-22T17:51:45.089ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
25102024-08-22T17:51:45.089ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
25112024-08-22T17:51:45.089ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
25122024-08-22T17:51:45.090ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-c961ede4-7532-4ca1-95a5-690b4d7c9b5a.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-c961ede4-7532-4ca1-95a5-690b4d7c9b5a.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1693, tv_nsec: 622284033 } })
25132024-08-22T17:51:45.090ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25142024-08-22T17:51:45.090ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1050
25152024-08-22T17:51:45.093ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
25162024-08-22T17:51:45.093ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
25172024-08-22T17:51:45.093ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
25182024-08-22T17:51:45.098ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
25192024-08-22T17:51:45.098ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
25202024-08-22T17:51:45.098ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25212024-08-22T17:51:45.098ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
25222024-08-22T17:51:45.098ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
25232024-08-22T17:51:45.101ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-c961ede4-7532-4ca1-95a5-690b4d7c9b5a.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-c961ede4-7532-4ca1-95a5-690b4d7c9b5a.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1693, tv_nsec: 633715512 } })
25242024-08-22T17:51:45.102ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25252024-08-22T17:51:45.103ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
25262024-08-22T17:51:45.103ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1050 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561
25272024-08-22T17:51:45.104ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
25282024-08-22T17:51:45.104ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
25292024-08-22T17:51:45.105ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
25302024-08-22T17:51:45.105ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
25312024-08-22T17:51:45.105ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
25322024-08-22T17:51:45.105ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
25332024-08-22T17:51:45.105ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
25342024-08-22T17:51:45.106ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
25352024-08-22T17:51:45.106ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
25362024-08-22T17:51:45.106ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
25372024-08-22T17:51:45.123ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25382024-08-22T17:51:45.128ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
25392024-08-22T17:51:45.128ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
25402024-08-22T17:51:45.128ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
25412024-08-22T17:51:45.128ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25422024-08-22T17:51:45.129ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0
25432024-08-22T17:51:45.129ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:189 found = 0 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0
25442024-08-22T17:51:45.129ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 60.337052ms iteration = 3 status = {"error":null,"error_count":0,"found":0,"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0}
25452024-08-22T17:51:45.132ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
25462024-08-22T17:51:45.133ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 7f16ce95-fbde-4896-9a22-7f61d8d01c5d dns_zone_name = oxide.test found_so_far = 0 version = 2
25472024-08-22T17:51:45.133ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
25482024-08-22T17:51:45.133ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
25492024-08-22T17:51:45.133ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
25502024-08-22T17:51:45.135ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25512024-08-22T17:51:45.136ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25522024-08-22T17:51:45.136ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
25532024-08-22T17:51:45.136ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
25542024-08-22T17:51:45.136ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
25552024-08-22T17:51:45.136ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
25562024-08-22T17:51:45.136ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
25572024-08-22T17:51:45.137ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
25582024-08-22T17:51:45.137ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
25592024-08-22T17:51:45.138ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
25602024-08-22T17:51:45.138ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
25612024-08-22T17:51:45.139ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
25622024-08-22T17:51:45.141ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
25632024-08-22T17:51:45.141ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
25642024-08-22T17:51:45.141ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
25652024-08-22T17:51:45.141ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25662024-08-22T17:51:45.146ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
25672024-08-22T17:51:45.146ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
25682024-08-22T17:51:45.146ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
25692024-08-22T17:51:45.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
25702024-08-22T17:51:45.147ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
25712024-08-22T17:51:45.149ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 77.735752ms iteration = 5 status = {"errors":[],"garbage_collect_requested":[]}
25722024-08-22T17:51:45.151ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
25732024-08-22T17:51:45.151ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
25742024-08-22T17:51:45.153ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
25752024-08-22T17:51:45.153ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = d6f65b4e-2e87-480a-af21-a69b4520828c dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
25762024-08-22T17:51:45.153ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
25772024-08-22T17:51:45.153ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 84.762018ms iteration = 5 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
25782024-08-22T17:51:45.154ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
25792024-08-22T17:51:45.154ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
25802024-08-22T17:51:45.154ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:140 found = 0 not_ready_to_be_deleted = 0
25812024-08-22T17:51:45.154ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25822024-08-22T17:51:45.154ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 84.822918ms iteration = 3 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
25832024-08-22T17:51:45.154ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
25842024-08-22T17:51:45.154ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
25852024-08-22T17:51:45.155ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25862024-08-22T17:51:45.155ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:55
25872024-08-22T17:51:45.155ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
25882024-08-22T17:51:45.155ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
25892024-08-22T17:51:45.156ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
25902024-08-22T17:51:45.156ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25912024-08-22T17:51:45.156ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25922024-08-22T17:51:45.156ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
25932024-08-22T17:51:45.156ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
25942024-08-22T17:51:45.156ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 87.645703ms iteration = 13 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
25952024-08-22T17:51:45.157ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
25962024-08-22T17:51:45.159ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
25972024-08-22T17:51:45.159ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
25982024-08-22T17:51:45.159ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
25992024-08-22T17:51:45.160ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: c961ede4-7532-4ca1-95a5-690b4d7c9b5a, lookup_type: ById(c961ede4-7532-4ca1-95a5-690b4d7c9b5a) } result = Ok(())
26002024-08-22T17:51:45.160ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
26012024-08-22T17:51:45.160ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
26022024-08-22T17:51:45.160ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
26032024-08-22T17:51:45.160ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
26042024-08-22T17:51:45.160ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
26052024-08-22T17:51:45.161ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26062024-08-22T17:51:45.163ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
26072024-08-22T17:51:45.163ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
26082024-08-22T17:51:45.164ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 95.091811ms iteration = 5 status = {"region_replacement_started_err":0,"region_replacement_started_ok":0}
26092024-08-22T17:51:45.165ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 147bdc10-344c-4ee8-a219-603cfc66e6d2, lookup_type: ById(147bdc10-344c-4ee8-a219-603cfc66e6d2) } result = Ok(())
26102024-08-22T17:51:45.165ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
26112024-08-22T17:51:45.165ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
26122024-08-22T17:51:45.168ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 98.951682ms iteration = 5 status = 0
26132024-08-22T17:51:45.176ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26142024-08-22T17:51:45.177ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 108.911653ms iteration = 3 status = {"errors":[],"found_port_ok":[]}
26152024-08-22T17:51:45.178ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
26162024-08-22T17:51:45.178ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
26172024-08-22T17:51:45.178ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
26182024-08-22T17:51:45.180ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 7f16ce95-fbde-4896-9a22-7f61d8d01c5d dns_zone_name = oxide.test found_so_far = 1 version = 2
26192024-08-22T17:51:45.180ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
26202024-08-22T17:51:45.180ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-08-22 17:49:58.502032 UTC dns_group = external generation = 2
26212024-08-22T17:51:45.180ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 111.157534ms iteration = 3 status = {"generation":2}
26222024-08-22T17:51:45.182ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 147bdc10-344c-4ee8-a219-603cfc66e6d2 original_time_created = 2024-08-22 17:49:18.445237 UTC target_id = 147bdc10-344c-4ee8-a219-603cfc66e6d2 time_created = 2024-08-22 17:49:18.445237 UTC
26232024-08-22T17:51:45.182ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 113.809558ms iteration = 13 status = {"status":"target blueprint unchanged","target_id":"147bdc10-344c-4ee8-a219-603cfc66e6d2","time_created":"2024-08-22T17:49:18.445237Z"}
26242024-08-22T17:51:45.194ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
26252024-08-22T17:51:45.195ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
26262024-08-22T17:51:45.195ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
26272024-08-22T17:51:45.198ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26282024-08-22T17:51:45.200ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
26292024-08-22T17:51:45.200ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
26302024-08-22T17:51:45.200ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
26312024-08-22T17:51:45.202ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26322024-08-22T17:51:45.204ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
26332024-08-22T17:51:45.204ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
26342024-08-22T17:51:45.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
26352024-08-22T17:51:45.208ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 138.894948ms iteration = 6 status = {"physical_disks_added":0}
26362024-08-22T17:51:45.212ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:55
26372024-08-22T17:51:45.212ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:220 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
26382024-08-22T17:51:45.212ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 142.599387ms iteration = 5 status = {"destroyed_active_vmms":0,"error":null,"saga_start_failures":0,"sagas_completed":0,"sagas_failed":0,"sagas_started":0,"terminated_active_migrations":0}
26392024-08-22T17:51:45.218ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:454 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
26402024-08-22T17:51:45.218ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 149.439802ms iteration = 5 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
26412024-08-22T17:51:45.221ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-08-22T17:49:43.060798Z, time_modified: 2024-08-22T17:49:43.060798Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 10d39b39-43d3-41db-b622-3ef3e09e2dad, db_silo: Silo { identity: SiloIdentity { id: 10d39b39-43d3-41db-b622-3ef3e09e2dad, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-08-22T17:49:58.442383Z, time_modified: 2024-08-22T17:49:58.442383Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "852177ddd7117113b778290a63910551c7b25a6718e9c8fda4590027c9443fee" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 10d39b39-43d3-41db-b622-3ef3e09e2dad, db_silo: Silo { identity: SiloIdentity { id: 10d39b39-43d3-41db-b622-3ef3e09e2dad, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-08-22T17:49:58.442383Z, time_modified: 2024-08-22T17:49:58.442383Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "852177ddd7117113b778290a63910551c7b25a6718e9c8fda4590027c9443fee" }] }) }
26422024-08-22T17:51:45.223ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 154.511179ms iteration = 3 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"10d39b39-43d3-41db-b622-3ef3e09e2dad","tls_certs":["852177ddd7117113b778290a63910551c7b25a6718e9c8fda4590027c9443fee"]}},"default_endpoint":{"silo_id":"10d39b39-43d3-41db-b622-3ef3e09e2dad","tls_certs":["852177ddd7117113b778290a63910551c7b25a6718e9c8fda4590027c9443fee"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
26432024-08-22T17:51:45.239ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
26442024-08-22T17:51:45.263ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1058 zone_address = fd00:1122:3344:101::2
26452024-08-22T17:51:45.263ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
26462024-08-22T17:51:45.263ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "ba222d94-8dca-4cd0-9e63-5d3d92242f85", "content-length": "26", "date": "Thu, 22 Aug 2024 17:51:44 GMT"} })
26472024-08-22T17:51:45.263ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1063 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
26482024-08-22T17:51:45.265ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1095 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
26492024-08-22T17:51:45.282ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = d6f65b4e-2e87-480a-af21-a69b4520828c dns_zone_name = control-plane.oxide.internal found_so_far = 50 version = 1
26502024-08-22T17:51:45.282ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
26512024-08-22T17:51:45.282ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-08-22 17:49:57.683168 UTC dns_group = internal generation = 1
26522024-08-22T17:51:45.282ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 213.354462ms iteration = 3 status = {"generation":1}
26532024-08-22T17:51:45.368ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
26542024-08-22T17:51:45.404ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1058 zone_address = fd00:1122:3344:101::2
26552024-08-22T17:51:45.404ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
26562024-08-22T17:51:45.404ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "3006c7a4-a525-45ef-a46b-e59085228c72", "content-length": "26", "date": "Thu, 22 Aug 2024 17:51:44 GMT"} })
26572024-08-22T17:51:45.404ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1063 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
26582024-08-22T17:51:45.405ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1095 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
26592024-08-22T17:51:45.407ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1058 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 zone_address = fd00:1122:3344:101::2
26602024-08-22T17:51:45.407ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
26612024-08-22T17:51:45.408ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "1a8b4497-5633-4cff-87c1-a9dc08c15fe0", "content-length": "26", "date": "Thu, 22 Aug 2024 17:51:44 GMT"} })
26622024-08-22T17:51:45.408ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1063 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
26632024-08-22T17:51:45.409ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1095 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561
26642024-08-22T17:51:45.585ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
26652024-08-22T17:51:45.585ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:3::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "bc8f4c2b-a649-4173-81ae-72aa8eb07fba", "date": "Thu, 22 Aug 2024 17:51:45 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
26662024-08-22T17:51:45.585ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
26672024-08-22T17:51:45.586ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:1::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "4743cf82-7187-4770-b443-2ce190310407", "date": "Thu, 22 Aug 2024 17:51:45 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
26682024-08-22T17:51:45.586ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
26692024-08-22T17:51:45.587ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:2::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "3e68a812-5ccc-4b4c-8697-05857bfd312f", "date": "Thu, 22 Aug 2024 17:51:45 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
26702024-08-22T17:51:45.587ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
26712024-08-22T17:51:45.589ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
26722024-08-22T17:51:45.589ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 520.043727ms iteration = 5 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
26732024-08-22T17:51:45.608ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 538.925715ms iteration = 5 status = {}
26742024-08-22T17:51:45.773ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
26752024-08-22T17:51:45.774ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "ce428e4d-99ff-48fe-871e-062049eaf3b3", "content-length": "1", "date": "Thu, 22 Aug 2024 17:51:45 GMT"} })
26762024-08-22T17:51:45.776ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
26772024-08-22T17:51:45.776ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
26782024-08-22T17:51:45.777ZDEBGnexus (BackgroundTasks): client request SledAgent = c961ede4-7532-4ca1-95a5-690b4d7c9b5a background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
26792024-08-22T17:51:45.778ZDEBGnexus (BackgroundTasks): client response SledAgent = c961ede4-7532-4ca1-95a5-690b4d7c9b5a background_task = vpc_route_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/vpc-routes", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "3d0e0570-0238-4893-94a2-7bbadc52458a", "content-length": "372", "date": "Thu, 22 Aug 2024 17:51:45 GMT"} })
26802024-08-22T17:51:45.778ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
26812024-08-22T17:51:45.778ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 709.021504ms iteration = 5 status = null
26822024-08-22T17:51:45.778ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
26832024-08-22T17:51:45.781ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
26842024-08-22T17:51:45.781ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
26852024-08-22T17:51:45.782ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
26862024-08-22T17:51:45.782ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::9)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "548d7e97-6e28-4e40-85bf-3d8daf15b742", "date": "Thu, 22 Aug 2024 17:51:45 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
26872024-08-22T17:51:45.782ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
26882024-08-22T17:51:45.783ZDEBGnexus (BackgroundTasks): client request SledAgent = c961ede4-7532-4ca1-95a5-690b4d7c9b5a background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
26892024-08-22T17:51:45.784ZDEBGnexus (BackgroundTasks): client response SledAgent = c961ede4-7532-4ca1-95a5-690b4d7c9b5a background_task = v2p_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/v2p", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "980925ef-e678-495f-ad10-feb9967fa392", "content-length": "691", "date": "Thu, 22 Aug 2024 17:51:45 GMT"} })
26902024-08-22T17:51:45.784ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = buskin
26912024-08-22T17:51:45.784ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = buskin
26922024-08-22T17:51:45.784ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = buskin
26932024-08-22T17:51:45.784ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
26942024-08-22T17:51:45.784ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
26952024-08-22T17:51:45.784ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::8)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "7be8f781-5292-4d1d-adc9-f0e267fa85f1", "date": "Thu, 22 Aug 2024 17:51:45 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
26962024-08-22T17:51:45.784ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
26972024-08-22T17:51:45.786ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 716.453863ms iteration = 5 status = {}
26982024-08-22T17:51:45.787ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
26992024-08-22T17:51:45.787ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 717.573148ms iteration = 5 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
27002024-08-22T17:51:45.788ZDEBGnexus (BackgroundTasks): client request SledAgent = c961ede4-7532-4ca1-95a5-690b4d7c9b5a background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
27012024-08-22T17:51:45.789ZDEBGnexus (BackgroundTasks): client response SledAgent = c961ede4-7532-4ca1-95a5-690b4d7c9b5a background_task = vpc_route_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/vpc-routes", query: None, fragment: None }, status: 204, headers: {"x-request-id": "fc08691e-3f04-4a87-b7bd-7d7f327d260a", "date": "Thu, 22 Aug 2024 17:51:45 GMT"} })
27022024-08-22T17:51:45.790ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 720.851305ms iteration = 5 status = {}
27032024-08-22T17:51:45.805ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
27042024-08-22T17:51:45.805ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
27052024-08-22T17:51:45.807ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:147 port = SwitchPort { id: c772d372-7448-4ce0-b8d1-efa645d18ed1, rack_id: c66d93a6-e2c3-4b19-9564-3b58bad50561, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(9ff05247-84aa-47d8-8619-ce0a8dbbaf7c) } rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 switch_location = Switch0
27062024-08-22T17:51:45.807ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
27072024-08-22T17:51:45.807ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
27082024-08-22T17:51:45.814ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
27092024-08-22T17:51:45.815ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/port/qsfp0/settings", query: Some("tag=omicron"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "bf1bb637-d50f-4f38-a9f2-89114a98ca4d", "content-length": "121", "date": "Thu, 22 Aug 2024 17:51:45 GMT"} })
27102024-08-22T17:51:45.815ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1714 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G } }} } switch_port_id = "qsfp0"
27112024-08-22T17:51:45.815ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1741 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 9ff05247-84aa-47d8-8619-ce0a8dbbaf7c
27122024-08-22T17:51:45.815ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:379 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
27132024-08-22T17:51:45.815ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
27142024-08-22T17:51:45.815ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:392 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 routes = {}
27152024-08-22T17:51:45.815ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:398 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 routes = {}
27162024-08-22T17:51:45.815ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:418 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561
27172024-08-22T17:51:45.815ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
27182024-08-22T17:51:45.815ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
27192024-08-22T17:51:45.817ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
27202024-08-22T17:51:45.817ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv4", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "10aad481-7126-465c-a62d-20e30b69caec", "content-length": "2", "date": "Thu, 22 Aug 2024 17:51:45 GMT"} })
27212024-08-22T17:51:45.817ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
27222024-08-22T17:51:45.817ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "02473932-6939-4e6c-87f7-31d8cf3297d2", "content-length": "52", "date": "Thu, 22 Aug 2024 17:51:45 GMT"} })
27232024-08-22T17:51:45.817ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0" }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:467 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 switch_location = Switch0
27242024-08-22T17:51:45.817ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
27252024-08-22T17:51:45.912ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 227 local_addr = [fd00:1122:3344:101::c]:56994 method = GET remote_addr = [fd00:1122:3344:101::d]:40670 req_id = 02445c3d-73e5-4b3f-bd3e-b020ea7b2600 response_code = 200 uri = /eaa260db-2b19-4ae1-97e8-5735c9f5d955
27262024-08-22T17:51:45.929ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/switch-ports", query: None, fragment: None }, status: 204, headers: {"x-request-id": "7eda8d14-6b0d-48d5-ba05-7473a2563187", "date": "Thu, 22 Aug 2024 17:51:45 GMT"} })
27272024-08-22T17:51:45.929ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:770 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 switch = Switch0
27282024-08-22T17:51:45.929ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
27292024-08-22T17:51:45.931ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/network-bootstore-config", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "228b6e70-af8a-4158-89e5-bc139919a7c9", "content-length": "667", "date": "Thu, 22 Aug 2024 17:51:45 GMT"} })
27302024-08-22T17:51:45.931ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
27312024-08-22T17:51:45.931ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
27322024-08-22T17:51:45.938ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
27332024-08-22T17:51:45.939ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
27342024-08-22T17:51:45.942ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
27352024-08-22T17:51:45.942ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
27362024-08-22T17:51:45.945ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
27372024-08-22T17:51:45.945ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
27382024-08-22T17:51:45.952ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 882.615731ms iteration = 5 status = {}
27392024-08-22T17:51:55.070ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 14 reason = Timeout
27402024-08-22T17:51:55.070ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 14 reason = Timeout
27412024-08-22T17:51:55.070ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
27422024-08-22T17:51:55.070ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
27432024-08-22T17:51:55.070ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
27442024-08-22T17:51:55.070ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
27452024-08-22T17:51:55.071ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27462024-08-22T17:51:55.072ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
27472024-08-22T17:51:55.072ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
27482024-08-22T17:51:55.074ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
27492024-08-22T17:51:55.074ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
27502024-08-22T17:51:55.074ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
27512024-08-22T17:51:55.074ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.371983ms iteration = 14 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
27522024-08-22T17:51:55.075ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
27532024-08-22T17:51:55.076ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
27542024-08-22T17:51:55.077ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27552024-08-22T17:51:55.080ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 147bdc10-344c-4ee8-a219-603cfc66e6d2, lookup_type: ById(147bdc10-344c-4ee8-a219-603cfc66e6d2) } result = Ok(())
27562024-08-22T17:51:55.080ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
27572024-08-22T17:51:55.080ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
27582024-08-22T17:51:55.085ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 147bdc10-344c-4ee8-a219-603cfc66e6d2 original_time_created = 2024-08-22 17:49:18.445237 UTC target_id = 147bdc10-344c-4ee8-a219-603cfc66e6d2 time_created = 2024-08-22 17:49:18.445237 UTC
27592024-08-22T17:51:55.085ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 15.781052ms iteration = 14 status = {"status":"target blueprint unchanged","target_id":"147bdc10-344c-4ee8-a219-603cfc66e6d2","time_created":"2024-08-22T17:49:18.445237Z"}
27602024-08-22T17:51:55.913ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 320 local_addr = [fd00:1122:3344:101::c]:56994 method = GET remote_addr = [fd00:1122:3344:101::d]:40670 req_id = b8f0775c-fa57-41df-8bb1-7dd60adc3e7f response_code = 200 uri = /eaa260db-2b19-4ae1-97e8-5735c9f5d955
27612024-08-22T17:52:05.070ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 15 reason = Timeout
27622024-08-22T17:52:05.070ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
27632024-08-22T17:52:05.070ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 15 reason = Timeout
27642024-08-22T17:52:05.070ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
27652024-08-22T17:52:05.070ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
27662024-08-22T17:52:05.071ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
27672024-08-22T17:52:05.072ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
27682024-08-22T17:52:05.072ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
27692024-08-22T17:52:05.073ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27702024-08-22T17:52:05.074ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.02745ms iteration = 15 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
27712024-08-22T17:52:05.075ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
27722024-08-22T17:52:05.075ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
27732024-08-22T17:52:05.075ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
27742024-08-22T17:52:05.077ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
27752024-08-22T17:52:05.077ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
27762024-08-22T17:52:05.079ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27772024-08-22T17:52:05.081ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 147bdc10-344c-4ee8-a219-603cfc66e6d2, lookup_type: ById(147bdc10-344c-4ee8-a219-603cfc66e6d2) } result = Ok(())
27782024-08-22T17:52:05.081ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
27792024-08-22T17:52:05.081ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
27802024-08-22T17:52:05.087ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 147bdc10-344c-4ee8-a219-603cfc66e6d2 original_time_created = 2024-08-22 17:49:18.445237 UTC target_id = 147bdc10-344c-4ee8-a219-603cfc66e6d2 time_created = 2024-08-22 17:49:18.445237 UTC
27812024-08-22T17:52:05.087ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 17.28001ms iteration = 15 status = {"status":"target blueprint unchanged","target_id":"147bdc10-344c-4ee8-a219-603cfc66e6d2","time_created":"2024-08-22T17:49:18.445237Z"}
27822024-08-22T17:52:05.913ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 231 local_addr = [fd00:1122:3344:101::c]:56994 method = GET remote_addr = [fd00:1122:3344:101::d]:40670 req_id = a6b7fb98-5fd7-4344-9459-e82813dc1152 response_code = 200 uri = /eaa260db-2b19-4ae1-97e8-5735c9f5d955
27832024-08-22T17:52:15.070ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 16 reason = Timeout
27842024-08-22T17:52:15.070ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 6 reason = Timeout
27852024-08-22T17:52:15.070ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
27862024-08-22T17:52:15.070ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
27872024-08-22T17:52:15.070ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 6 reason = Timeout
27882024-08-22T17:52:15.070ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 6 reason = Timeout
27892024-08-22T17:52:15.070ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
27902024-08-22T17:52:15.070ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 6 reason = Timeout
27912024-08-22T17:52:15.070ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
27922024-08-22T17:52:15.070ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
27932024-08-22T17:52:15.070ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
27942024-08-22T17:52:15.070ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 6 reason = Timeout
27952024-08-22T17:52:15.070ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
27962024-08-22T17:52:15.070ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 6 reason = Timeout
27972024-08-22T17:52:15.070ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
27982024-08-22T17:52:15.070ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 6 reason = Timeout
27992024-08-22T17:52:15.070ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 6 reason = Timeout
28002024-08-22T17:52:15.070ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
28012024-08-22T17:52:15.070ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
28022024-08-22T17:52:15.070ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 6 reason = Timeout
28032024-08-22T17:52:15.070ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
28042024-08-22T17:52:15.070ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 7 reason = Timeout
28052024-08-22T17:52:15.070ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 6 reason = Timeout
28062024-08-22T17:52:15.070ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
28072024-08-22T17:52:15.070ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
28082024-08-22T17:52:15.071ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 16 reason = Timeout
28092024-08-22T17:52:15.071ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
28102024-08-22T17:52:15.071ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 6 reason = Timeout
28112024-08-22T17:52:15.071ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
28122024-08-22T17:52:15.071ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
28132024-08-22T17:52:15.071ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
28142024-08-22T17:52:15.071ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
28152024-08-22T17:52:15.071ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
28162024-08-22T17:52:15.071ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
28172024-08-22T17:52:15.071ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
28182024-08-22T17:52:15.071ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
28192024-08-22T17:52:15.071ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
28202024-08-22T17:52:15.071ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
28212024-08-22T17:52:15.071ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
28222024-08-22T17:52:15.071ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
28232024-08-22T17:52:15.071ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
28242024-08-22T17:52:15.072ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 6 reason = Timeout
28252024-08-22T17:52:15.072ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
28262024-08-22T17:52:15.073ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
28272024-08-22T17:52:15.073ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28282024-08-22T17:52:15.073ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 2.933055ms iteration = 6 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
28292024-08-22T17:52:15.074ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28302024-08-22T17:52:15.075ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
28312024-08-22T17:52:15.076ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
28322024-08-22T17:52:15.076ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
28332024-08-22T17:52:15.076ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
28342024-08-22T17:52:15.076ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
28352024-08-22T17:52:15.076ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
28362024-08-22T17:52:15.076ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
28372024-08-22T17:52:15.077ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
28382024-08-22T17:52:15.078ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28392024-08-22T17:52:15.079ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-c961ede4-7532-4ca1-95a5-690b4d7c9b5a.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-c961ede4-7532-4ca1-95a5-690b4d7c9b5a.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1723, tv_nsec: 610066703 } })
28402024-08-22T17:52:15.079ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
28412024-08-22T17:52:15.079ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1050
28422024-08-22T17:52:15.080ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
28432024-08-22T17:52:15.080ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28442024-08-22T17:52:15.081ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-c961ede4-7532-4ca1-95a5-690b4d7c9b5a.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-c961ede4-7532-4ca1-95a5-690b4d7c9b5a.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1723, tv_nsec: 611676781 } })
28452024-08-22T17:52:15.081ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28462024-08-22T17:52:15.081ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1050
28472024-08-22T17:52:15.081ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
28482024-08-22T17:52:15.081ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
28492024-08-22T17:52:15.082ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
28502024-08-22T17:52:15.083ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28512024-08-22T17:52:15.083ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 10.930417ms iteration = 6 status = {"errors":[],"garbage_collect_requested":[]}
28522024-08-22T17:52:15.084ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
28532024-08-22T17:52:15.084ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
28542024-08-22T17:52:15.084ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
28552024-08-22T17:52:15.085ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
28562024-08-22T17:52:15.085ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
28572024-08-22T17:52:15.085ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
28582024-08-22T17:52:15.086ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
28592024-08-22T17:52:15.086ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
28602024-08-22T17:52:15.087ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
28612024-08-22T17:52:15.090ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 19.807312ms iteration = 16 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
28622024-08-22T17:52:15.090ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
28632024-08-22T17:52:15.091ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
28642024-08-22T17:52:15.092ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 21.907394ms iteration = 6 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
28652024-08-22T17:52:15.093ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28662024-08-22T17:52:15.094ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28672024-08-22T17:52:15.094ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
28682024-08-22T17:52:15.094ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-c961ede4-7532-4ca1-95a5-690b4d7c9b5a.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-c961ede4-7532-4ca1-95a5-690b4d7c9b5a.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1723, tv_nsec: 625098231 } })
28692024-08-22T17:52:15.094ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
28702024-08-22T17:52:15.095ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
28712024-08-22T17:52:15.095ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
28722024-08-22T17:52:15.096ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
28732024-08-22T17:52:15.096ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
28742024-08-22T17:52:15.096ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1050 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561
28752024-08-22T17:52:15.096ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
28762024-08-22T17:52:15.097ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 147bdc10-344c-4ee8-a219-603cfc66e6d2, lookup_type: ById(147bdc10-344c-4ee8-a219-603cfc66e6d2) } result = Ok(())
28772024-08-22T17:52:15.097ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
28782024-08-22T17:52:15.097ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
28792024-08-22T17:52:15.098ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28802024-08-22T17:52:15.100ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
28812024-08-22T17:52:15.100ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
28822024-08-22T17:52:15.101ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
28832024-08-22T17:52:15.106ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 147bdc10-344c-4ee8-a219-603cfc66e6d2 original_time_created = 2024-08-22 17:49:18.445237 UTC target_id = 147bdc10-344c-4ee8-a219-603cfc66e6d2 time_created = 2024-08-22 17:49:18.445237 UTC
28842024-08-22T17:52:15.106ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 35.805365ms iteration = 16 status = {"status":"target blueprint unchanged","target_id":"147bdc10-344c-4ee8-a219-603cfc66e6d2","time_created":"2024-08-22T17:49:18.445237Z"}
28852024-08-22T17:52:15.117ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
28862024-08-22T17:52:15.118ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
28872024-08-22T17:52:15.123ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
28882024-08-22T17:52:15.124ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
28892024-08-22T17:52:15.127ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:55
28902024-08-22T17:52:15.127ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
28912024-08-22T17:52:15.127ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
28922024-08-22T17:52:15.127ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
28932024-08-22T17:52:15.150ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:454 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
28942024-08-22T17:52:15.150ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 80.229634ms iteration = 6 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
28952024-08-22T17:52:15.167ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1058 zone_address = fd00:1122:3344:101::2
28962024-08-22T17:52:15.167ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
28972024-08-22T17:52:15.167ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "5be7ab0e-c685-4aae-8d4e-ce9453e0cb9c", "content-length": "26", "date": "Thu, 22 Aug 2024 17:52:14 GMT"} })
28982024-08-22T17:52:15.167ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1063 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
28992024-08-22T17:52:15.169ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1095 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
29002024-08-22T17:52:15.176ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1058 zone_address = fd00:1122:3344:101::2
29012024-08-22T17:52:15.176ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
29022024-08-22T17:52:15.176ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "d5316799-f8c9-4410-a9bb-a28feae71520", "content-length": "26", "date": "Thu, 22 Aug 2024 17:52:14 GMT"} })
29032024-08-22T17:52:15.176ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1063 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
29042024-08-22T17:52:15.177ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1095 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
29052024-08-22T17:52:15.197ZDEBGnexus (BackgroundTasks): client request SledAgent = c961ede4-7532-4ca1-95a5-690b4d7c9b5a background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
29062024-08-22T17:52:15.199ZDEBGnexus (BackgroundTasks): client response SledAgent = c961ede4-7532-4ca1-95a5-690b4d7c9b5a background_task = vpc_route_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/vpc-routes", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "f03ca89d-a4ad-461d-a018-802cb0d2d2a6", "content-length": "372", "date": "Thu, 22 Aug 2024 17:52:15 GMT"} })
29072024-08-22T17:52:15.199ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
29082024-08-22T17:52:15.199ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
29092024-08-22T17:52:15.202ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
29102024-08-22T17:52:15.203ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
29112024-08-22T17:52:15.208ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
29122024-08-22T17:52:15.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
29132024-08-22T17:52:15.213ZDEBGnexus (BackgroundTasks): client request SledAgent = c961ede4-7532-4ca1-95a5-690b4d7c9b5a background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
29142024-08-22T17:52:15.214ZDEBGnexus (BackgroundTasks): client response SledAgent = c961ede4-7532-4ca1-95a5-690b4d7c9b5a background_task = vpc_route_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/vpc-routes", query: None, fragment: None }, status: 204, headers: {"x-request-id": "681d66b3-ac3b-4e88-8fcd-efe74b82708b", "date": "Thu, 22 Aug 2024 17:52:15 GMT"} })
29152024-08-22T17:52:15.216ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 145.21382ms iteration = 6 status = {}
29162024-08-22T17:52:15.218ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1058 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 zone_address = fd00:1122:3344:101::2
29172024-08-22T17:52:15.218ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
29182024-08-22T17:52:15.218ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "45ef9fe9-1964-4988-9e9d-218720aed015", "content-length": "26", "date": "Thu, 22 Aug 2024 17:52:14 GMT"} })
29192024-08-22T17:52:15.218ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1063 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
29202024-08-22T17:52:15.220ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1095 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561
29212024-08-22T17:52:15.237ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 166.755562ms iteration = 6 status = {"region_replacement_started_err":0,"region_replacement_started_ok":0}
29222024-08-22T17:52:15.238ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 167.851188ms iteration = 7 status = {"physical_disks_added":0}
29232024-08-22T17:52:15.239ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:55
29242024-08-22T17:52:15.239ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:220 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
29252024-08-22T17:52:15.239ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 169.214825ms iteration = 6 status = {"destroyed_active_vmms":0,"error":null,"saga_start_failures":0,"sagas_completed":0,"sagas_failed":0,"sagas_started":0,"terminated_active_migrations":0}
29262024-08-22T17:52:15.270ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
29272024-08-22T17:52:15.270ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "c862e81b-0f1d-45d1-a6e9-2f601d97ab87", "content-length": "1", "date": "Thu, 22 Aug 2024 17:52:14 GMT"} })
29282024-08-22T17:52:15.271ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
29292024-08-22T17:52:15.271ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
29302024-08-22T17:52:15.274ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 203.24934ms iteration = 6 status = null
29312024-08-22T17:52:15.285ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 214.67069ms iteration = 6 status = {}
29322024-08-22T17:52:15.292ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
29332024-08-22T17:52:15.292ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
29342024-08-22T17:52:15.294ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29352024-08-22T17:52:15.295ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
29362024-08-22T17:52:15.295ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
29372024-08-22T17:52:15.295ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
29382024-08-22T17:52:15.326ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29392024-08-22T17:52:15.328ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
29402024-08-22T17:52:15.328ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
29412024-08-22T17:52:15.328ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
29422024-08-22T17:52:15.381ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
29432024-08-22T17:52:15.381ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
29442024-08-22T17:52:15.384ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:147 port = SwitchPort { id: c772d372-7448-4ce0-b8d1-efa645d18ed1, rack_id: c66d93a6-e2c3-4b19-9564-3b58bad50561, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(9ff05247-84aa-47d8-8619-ce0a8dbbaf7c) } rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 switch_location = Switch0
29452024-08-22T17:52:15.384ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
29462024-08-22T17:52:15.384ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
29472024-08-22T17:52:15.386ZDEBGnexus (BackgroundTasks): client request SledAgent = c961ede4-7532-4ca1-95a5-690b4d7c9b5a background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
29482024-08-22T17:52:15.387ZDEBGnexus (BackgroundTasks): client response SledAgent = c961ede4-7532-4ca1-95a5-690b4d7c9b5a background_task = v2p_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/v2p", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "c9f633eb-b7d6-4d94-b7eb-847254f48887", "content-length": "691", "date": "Thu, 22 Aug 2024 17:52:15 GMT"} })
29492024-08-22T17:52:15.387ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = buskin
29502024-08-22T17:52:15.387ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = buskin
29512024-08-22T17:52:15.387ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = buskin
29522024-08-22T17:52:15.389ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 318.333445ms iteration = 6 status = {}
29532024-08-22T17:52:15.484ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
29542024-08-22T17:52:15.485ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/port/qsfp0/settings", query: Some("tag=omicron"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "6806e216-5187-443f-a9e4-40bb7e3a170b", "content-length": "121", "date": "Thu, 22 Aug 2024 17:52:15 GMT"} })
29552024-08-22T17:52:15.485ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1714 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G } }} } switch_port_id = "qsfp0"
29562024-08-22T17:52:15.485ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1741 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 9ff05247-84aa-47d8-8619-ce0a8dbbaf7c
29572024-08-22T17:52:15.485ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:379 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
29582024-08-22T17:52:15.485ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
29592024-08-22T17:52:15.485ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:392 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 routes = {}
29602024-08-22T17:52:15.485ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:398 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 routes = {}
29612024-08-22T17:52:15.485ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:418 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561
29622024-08-22T17:52:15.485ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
29632024-08-22T17:52:15.486ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
29642024-08-22T17:52:15.487ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
29652024-08-22T17:52:15.487ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv4", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "2f03ce09-d6a3-4f11-aa8a-968355397b72", "content-length": "2", "date": "Thu, 22 Aug 2024 17:52:15 GMT"} })
29662024-08-22T17:52:15.487ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
29672024-08-22T17:52:15.487ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "70f2aeb4-f394-4e28-bdb2-b59ae1201279", "content-length": "52", "date": "Thu, 22 Aug 2024 17:52:15 GMT"} })
29682024-08-22T17:52:15.487ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0" }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:467 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 switch_location = Switch0
29692024-08-22T17:52:15.487ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
29702024-08-22T17:52:15.549ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
29712024-08-22T17:52:15.549ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
29722024-08-22T17:52:15.552ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
29732024-08-22T17:52:15.552ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
29742024-08-22T17:52:15.599ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29752024-08-22T17:52:15.605ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: c961ede4-7532-4ca1-95a5-690b4d7c9b5a, lookup_type: ById(c961ede4-7532-4ca1-95a5-690b4d7c9b5a) } result = Ok(())
29762024-08-22T17:52:15.605ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
29772024-08-22T17:52:15.605ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
29782024-08-22T17:52:15.607ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 536.859325ms iteration = 6 status = 0
29792024-08-22T17:52:15.617ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/switch-ports", query: None, fragment: None }, status: 204, headers: {"x-request-id": "626d6234-82b1-4a2b-ad3a-8904c33f12e2", "date": "Thu, 22 Aug 2024 17:52:15 GMT"} })
29802024-08-22T17:52:15.617ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:770 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 switch = Switch0
29812024-08-22T17:52:15.617ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
29822024-08-22T17:52:15.617ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/network-bootstore-config", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "265d08b5-1ecf-4f4b-b0b8-36526fe669b2", "content-length": "667", "date": "Thu, 22 Aug 2024 17:52:15 GMT"} })
29832024-08-22T17:52:15.617ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
29842024-08-22T17:52:15.618ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
29852024-08-22T17:52:15.621ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
29862024-08-22T17:52:15.621ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
29872024-08-22T17:52:15.624ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
29882024-08-22T17:52:15.624ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
29892024-08-22T17:52:15.626ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
29902024-08-22T17:52:15.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
29912024-08-22T17:52:15.633ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 562.415176ms iteration = 6 status = {}
29922024-08-22T17:52:15.914ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 277 local_addr = [fd00:1122:3344:101::c]:56994 method = GET remote_addr = [fd00:1122:3344:101::d]:40670 req_id = 71f65ed1-bbc9-400c-9479-5413e9453633 response_code = 200 uri = /eaa260db-2b19-4ae1-97e8-5735c9f5d955
29932024-08-22T17:52:25.071ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 17 reason = Timeout
29942024-08-22T17:52:25.071ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
29952024-08-22T17:52:25.071ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 17 reason = Timeout
29962024-08-22T17:52:25.071ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
29972024-08-22T17:52:25.071ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
29982024-08-22T17:52:25.071ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
29992024-08-22T17:52:25.073ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
30002024-08-22T17:52:25.073ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30012024-08-22T17:52:25.073ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
30022024-08-22T17:52:25.076ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
30032024-08-22T17:52:25.076ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
30042024-08-22T17:52:25.076ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
30052024-08-22T17:52:25.078ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
30062024-08-22T17:52:25.078ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
30072024-08-22T17:52:25.079ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30082024-08-22T17:52:25.082ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 147bdc10-344c-4ee8-a219-603cfc66e6d2, lookup_type: ById(147bdc10-344c-4ee8-a219-603cfc66e6d2) } result = Ok(())
30092024-08-22T17:52:25.082ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
30102024-08-22T17:52:25.082ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
30112024-08-22T17:52:25.091ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 147bdc10-344c-4ee8-a219-603cfc66e6d2 original_time_created = 2024-08-22 17:49:18.445237 UTC target_id = 147bdc10-344c-4ee8-a219-603cfc66e6d2 time_created = 2024-08-22 17:49:18.445237 UTC
30122024-08-22T17:52:25.091ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 19.51229ms iteration = 17 status = {"status":"target blueprint unchanged","target_id":"147bdc10-344c-4ee8-a219-603cfc66e6d2","time_created":"2024-08-22T17:49:18.445237Z"}
30132024-08-22T17:52:25.099ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 28.120936ms iteration = 17 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
30142024-08-22T17:52:25.913ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 320 local_addr = [fd00:1122:3344:101::c]:56994 method = GET remote_addr = [fd00:1122:3344:101::d]:40670 req_id = cc78c758-99c3-4c0c-bdd7-99f303475e48 response_code = 200 uri = /eaa260db-2b19-4ae1-97e8-5735c9f5d955
30152024-08-22T17:52:35.071ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 18 reason = Timeout
30162024-08-22T17:52:35.071ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 18 reason = Timeout
30172024-08-22T17:52:35.071ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
30182024-08-22T17:52:35.071ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
30192024-08-22T17:52:35.072ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
30202024-08-22T17:52:35.072ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
30212024-08-22T17:52:35.073ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30222024-08-22T17:52:35.074ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
30232024-08-22T17:52:35.074ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
30242024-08-22T17:52:35.075ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
30252024-08-22T17:52:35.075ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
30262024-08-22T17:52:35.076ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
30272024-08-22T17:52:35.077ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
30282024-08-22T17:52:35.077ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
30292024-08-22T17:52:35.079ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30302024-08-22T17:52:35.081ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 147bdc10-344c-4ee8-a219-603cfc66e6d2, lookup_type: ById(147bdc10-344c-4ee8-a219-603cfc66e6d2) } result = Ok(())
30312024-08-22T17:52:35.081ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
30322024-08-22T17:52:35.081ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
30332024-08-22T17:52:35.089ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 147bdc10-344c-4ee8-a219-603cfc66e6d2 original_time_created = 2024-08-22 17:49:18.445237 UTC target_id = 147bdc10-344c-4ee8-a219-603cfc66e6d2 time_created = 2024-08-22 17:49:18.445237 UTC
30342024-08-22T17:52:35.089ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 17.958883ms iteration = 18 status = {"status":"target blueprint unchanged","target_id":"147bdc10-344c-4ee8-a219-603cfc66e6d2","time_created":"2024-08-22T17:49:18.445237Z"}
30352024-08-22T17:52:35.105ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 33.282721ms iteration = 18 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
30362024-08-22T17:52:35.914ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 327 local_addr = [fd00:1122:3344:101::c]:56994 method = GET remote_addr = [fd00:1122:3344:101::d]:40670 req_id = 5f10d26b-2d1f-4fd3-bbad-6ccb3ab9642c response_code = 200 uri = /eaa260db-2b19-4ae1-97e8-5735c9f5d955
30372024-08-22T17:52:45.072ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 4 reason = Timeout
30382024-08-22T17:52:45.072ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 4 reason = Timeout
30392024-08-22T17:52:45.072ZINFOnexus (BackgroundTasks): Abandoned VMM reaper running background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:76
30402024-08-22T17:52:45.072ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 6 reason = Timeout
30412024-08-22T17:52:45.072ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
30422024-08-22T17:52:45.072ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 7 reason = Timeout
30432024-08-22T17:52:45.072ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 6 reason = Timeout
30442024-08-22T17:52:45.072ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
30452024-08-22T17:52:45.072ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 7 reason = Timeout
30462024-08-22T17:52:45.072ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
30472024-08-22T17:52:45.072ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 7 reason = Timeout
30482024-08-22T17:52:45.072ZDEBGnexus (BackgroundTasks): activating background_task = crdb_node_id_collector iteration = 3 reason = Timeout
30492024-08-22T17:52:45.072ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
30502024-08-22T17:52:45.072ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
30512024-08-22T17:52:45.072ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 7 reason = Timeout
30522024-08-22T17:52:45.072ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
30532024-08-22T17:52:45.072ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 7 reason = Timeout
30542024-08-22T17:52:45.072ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
30552024-08-22T17:52:45.072ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 8 reason = Timeout
30562024-08-22T17:52:45.072ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 4 reason = Timeout
30572024-08-22T17:52:45.072ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
30582024-08-22T17:52:45.072ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2024-08-22 17:42:45.072268008 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
30592024-08-22T17:52:45.072ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
30602024-08-22T17:52:45.072ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 5 reason = Timeout
30612024-08-22T17:52:45.072ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:80 reason = blueprint disabled target_id = 147bdc10-344c-4ee8-a219-603cfc66e6d2
30622024-08-22T17:52:45.072ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 10.63\u{b5}s iteration = 5 status = {"error":"blueprint disabled","target_id":"147bdc10-344c-4ee8-a219-603cfc66e6d2"}
30632024-08-22T17:52:45.072ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 4 reason = Timeout
30642024-08-22T17:52:45.072ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 7 reason = Timeout
30652024-08-22T17:52:45.072ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
30662024-08-22T17:52:45.072ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
30672024-08-22T17:52:45.072ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 7 reason = Timeout
30682024-08-22T17:52:45.072ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 7 reason = Timeout
30692024-08-22T17:52:45.072ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
30702024-08-22T17:52:45.072ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 19 reason = Timeout
30712024-08-22T17:52:45.072ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
30722024-08-22T17:52:45.072ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
30732024-08-22T17:52:45.072ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 4 reason = Timeout
30742024-08-22T17:52:45.072ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:56
30752024-08-22T17:52:45.072ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
30762024-08-22T17:52:45.072ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 7 reason = Timeout
30772024-08-22T17:52:45.072ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 19 reason = Timeout
30782024-08-22T17:52:45.072ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 4 reason = Timeout
30792024-08-22T17:52:45.072ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 4 reason = Timeout
30802024-08-22T17:52:45.072ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
30812024-08-22T17:52:45.072ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
30822024-08-22T17:52:45.072ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
30832024-08-22T17:52:45.072ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
30842024-08-22T17:52:45.072ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 4 reason = Timeout
30852024-08-22T17:52:45.072ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
30862024-08-22T17:52:45.072ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
30872024-08-22T17:52:45.072ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 4 reason = Timeout
30882024-08-22T17:52:45.072ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
30892024-08-22T17:52:45.072ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 7 reason = Timeout
30902024-08-22T17:52:45.072ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
30912024-08-22T17:52:45.072ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 7 reason = Timeout
30922024-08-22T17:52:45.072ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
30932024-08-22T17:52:45.072ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
30942024-08-22T17:52:45.072ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
30952024-08-22T17:52:45.072ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
30962024-08-22T17:52:45.072ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
30972024-08-22T17:52:45.072ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
30982024-08-22T17:52:45.072ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
30992024-08-22T17:52:45.072ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
31002024-08-22T17:52:45.072ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
31012024-08-22T17:52:45.072ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
31022024-08-22T17:52:45.072ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
31032024-08-22T17:52:45.072ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
31042024-08-22T17:52:45.072ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
31052024-08-22T17:52:45.072ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
31062024-08-22T17:52:45.072ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
31072024-08-22T17:52:45.073ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
31082024-08-22T17:52:45.073ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
31092024-08-22T17:52:45.073ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
31102024-08-22T17:52:45.073ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
31112024-08-22T17:52:45.073ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
31122024-08-22T17:52:45.073ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
31132024-08-22T17:52:45.073ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
31142024-08-22T17:52:45.073ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
31152024-08-22T17:52:45.073ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
31162024-08-22T17:52:45.073ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
31172024-08-22T17:52:45.073ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
31182024-08-22T17:52:45.073ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
31192024-08-22T17:52:45.073ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
31202024-08-22T17:52:45.073ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
31212024-08-22T17:52:45.073ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
31222024-08-22T17:52:45.073ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 7 reason = Timeout
31232024-08-22T17:52:45.073ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
31242024-08-22T17:52:45.073ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("6507c79b-9a18-46c7-930d-c98e8662b87f.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("a09ea923-b85c-428d-b289-6936a0c2542c.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("c0ef020c-52c1-4b9a-a085-445fab31264f.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("c0ef020c-52c1-4b9a-a085-445fab31264f.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:3::1))) }], valid_until: Instant { tv_sec: 1753, tv_nsec: 603559212 } })
31252024-08-22T17:52:45.073ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("9488ce0a-6732-4f75-baf9-4ecfa406df91.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("afac2eab-cadf-48d0-94a7-ad562ade88c6.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("afac2eab-cadf-48d0-94a7-ad562ade88c6.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::9))) }], valid_until: Instant { tv_sec: 1753, tv_nsec: 603588112 } })
31262024-08-22T17:52:45.073ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
31272024-08-22T17:52:45.075ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31282024-08-22T17:52:45.076ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 4.543764ms iteration = 7 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
31292024-08-22T17:52:45.077ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31302024-08-22T17:52:45.079ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
31312024-08-22T17:52:45.079ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 6.980126ms iteration = 4 status = {"addresses":["[fd00:1122:3344:101::8]:5353","[fd00:1122:3344:101::9]:5353"]}
31322024-08-22T17:52:45.079ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
31332024-08-22T17:52:45.079ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
31342024-08-22T17:52:45.079ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
31352024-08-22T17:52:45.080ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
31362024-08-22T17:52:45.080ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
31372024-08-22T17:52:45.080ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
31382024-08-22T17:52:45.080ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } dns_group = internal
31392024-08-22T17:52:45.080ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 8.383053ms iteration = 4 status = {"addresses":["[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:3::1]:5353"]}
31402024-08-22T17:52:45.082ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-c961ede4-7532-4ca1-95a5-690b4d7c9b5a.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-c961ede4-7532-4ca1-95a5-690b4d7c9b5a.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1753, tv_nsec: 612121026 } })
31412024-08-22T17:52:45.082ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
31422024-08-22T17:52:45.084ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1050
31432024-08-22T17:52:45.084ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
31442024-08-22T17:52:45.084ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31452024-08-22T17:52:45.084ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2024-08-22 17:42:45.072268008 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
31462024-08-22T17:52:45.084ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 12.409194ms iteration = 4 status = {"expiration":"2024-08-22T17:42:45.072268008Z","pruned":[]}
31472024-08-22T17:52:45.086ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
31482024-08-22T17:52:45.086ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
31492024-08-22T17:52:45.086ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
31502024-08-22T17:52:45.088ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
31512024-08-22T17:52:45.089ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
31522024-08-22T17:52:45.090ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 17.671591ms iteration = 4 status = {"errors":[],"found_port_ok":[]}
31532024-08-22T17:52:45.092ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31542024-08-22T17:52:45.096ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
31552024-08-22T17:52:45.096ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
31562024-08-22T17:52:45.096ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
31572024-08-22T17:52:45.096ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31582024-08-22T17:52:45.099ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
31592024-08-22T17:52:45.099ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
31602024-08-22T17:52:45.099ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
31612024-08-22T17:52:45.102ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
31622024-08-22T17:52:45.103ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
31632024-08-22T17:52:45.107ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31642024-08-22T17:52:45.110ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
31652024-08-22T17:52:45.110ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
31662024-08-22T17:52:45.110ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
31672024-08-22T17:52:45.112ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31682024-08-22T17:52:45.118ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
31692024-08-22T17:52:45.120ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31702024-08-22T17:52:45.123ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
31712024-08-22T17:52:45.123ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
31722024-08-22T17:52:45.125ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
31732024-08-22T17:52:45.132ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31742024-08-22T17:52:45.135ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
31752024-08-22T17:52:45.135ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
31762024-08-22T17:52:45.135ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
31772024-08-22T17:52:45.136ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31782024-08-22T17:52:45.138ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31792024-08-22T17:52:45.139ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
31802024-08-22T17:52:45.139ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
31812024-08-22T17:52:45.139ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
31822024-08-22T17:52:45.140ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
31832024-08-22T17:52:45.140ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
31842024-08-22T17:52:45.141ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
31852024-08-22T17:52:45.142ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31862024-08-22T17:52:45.144ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
31872024-08-22T17:52:45.144ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
31882024-08-22T17:52:45.144ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
31892024-08-22T17:52:45.144ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31902024-08-22T17:52:45.145ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31912024-08-22T17:52:45.145ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-c961ede4-7532-4ca1-95a5-690b4d7c9b5a.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-c961ede4-7532-4ca1-95a5-690b4d7c9b5a.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1753, tv_nsec: 675348873 } })
31922024-08-22T17:52:45.145ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 72.331253ms iteration = 7 status = {"errors":[],"garbage_collect_requested":[]}
31932024-08-22T17:52:45.146ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
31942024-08-22T17:52:45.146ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
31952024-08-22T17:52:45.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
31962024-08-22T17:52:45.146ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
31972024-08-22T17:52:45.147ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
31982024-08-22T17:52:45.147ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1050 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561
31992024-08-22T17:52:45.148ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32002024-08-22T17:52:45.148ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
32012024-08-22T17:52:45.148ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
32022024-08-22T17:52:45.148ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
32032024-08-22T17:52:45.148ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-c961ede4-7532-4ca1-95a5-690b4d7c9b5a.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-c961ede4-7532-4ca1-95a5-690b4d7c9b5a.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1753, tv_nsec: 678484849 } })
32042024-08-22T17:52:45.149ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0
32052024-08-22T17:52:45.149ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:189 found = 0 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0
32062024-08-22T17:52:45.149ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 77.032778ms iteration = 4 status = {"error":null,"error_count":0,"found":0,"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0}
32072024-08-22T17:52:45.149ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1050
32082024-08-22T17:52:45.150ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
32092024-08-22T17:52:45.150ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
32102024-08-22T17:52:45.150ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:55
32112024-08-22T17:52:45.150ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
32122024-08-22T17:52:45.150ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
32132024-08-22T17:52:45.150ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
32142024-08-22T17:52:45.151ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32152024-08-22T17:52:45.151ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
32162024-08-22T17:52:45.152ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
32172024-08-22T17:52:45.152ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
32182024-08-22T17:52:45.153ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
32192024-08-22T17:52:45.153ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
32202024-08-22T17:52:45.153ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
32212024-08-22T17:52:45.153ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
32222024-08-22T17:52:45.153ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
32232024-08-22T17:52:45.162ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32242024-08-22T17:52:45.164ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
32252024-08-22T17:52:45.164ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
32262024-08-22T17:52:45.164ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
32272024-08-22T17:52:45.171ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
32282024-08-22T17:52:45.171ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
32292024-08-22T17:52:45.171ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
32302024-08-22T17:52:45.171ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
32312024-08-22T17:52:45.172ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
32322024-08-22T17:52:45.177ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
32332024-08-22T17:52:45.177ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
32342024-08-22T17:52:45.177ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
32352024-08-22T17:52:45.177ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
32362024-08-22T17:52:45.177ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
32372024-08-22T17:52:45.195ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 123.240087ms iteration = 7 status = {"region_replacement_started_err":0,"region_replacement_started_ok":0}
32382024-08-22T17:52:45.195ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
32392024-08-22T17:52:45.196ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:140 found = 0 not_ready_to_be_deleted = 0
32402024-08-22T17:52:45.196ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 123.614439ms iteration = 4 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
32412024-08-22T17:52:45.196ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 123.87786ms iteration = 7 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
32422024-08-22T17:52:45.199ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:55
32432024-08-22T17:52:45.199ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:220 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
32442024-08-22T17:52:45.199ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 127.494459ms iteration = 7 status = {"destroyed_active_vmms":0,"error":null,"saga_start_failures":0,"sagas_completed":0,"sagas_failed":0,"sagas_started":0,"terminated_active_migrations":0}
32452024-08-22T17:52:45.226ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
32462024-08-22T17:52:45.227ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
32472024-08-22T17:52:45.245ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 173.461176ms iteration = 8 status = {"physical_disks_added":0}
32482024-08-22T17:52:45.264ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
32492024-08-22T17:52:45.303ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
32502024-08-22T17:52:45.340ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1058 zone_address = fd00:1122:3344:101::2
32512024-08-22T17:52:45.340ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
32522024-08-22T17:52:45.340ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "e5aeec35-82d8-4c74-94e4-98e25dcf1084", "content-length": "26", "date": "Thu, 22 Aug 2024 17:52:44 GMT"} })
32532024-08-22T17:52:45.341ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1063 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
32542024-08-22T17:52:45.342ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1095 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
32552024-08-22T17:52:45.422ZDEBGnexus (BackgroundTasks): client request SledAgent = c961ede4-7532-4ca1-95a5-690b4d7c9b5a background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
32562024-08-22T17:52:45.423ZDEBGnexus (BackgroundTasks): client response SledAgent = c961ede4-7532-4ca1-95a5-690b4d7c9b5a background_task = vpc_route_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/vpc-routes", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "c76a74ff-04b9-44fa-8099-4221c93c01d6", "content-length": "372", "date": "Thu, 22 Aug 2024 17:52:44 GMT"} })
32572024-08-22T17:52:45.423ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
32582024-08-22T17:52:45.423ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
32592024-08-22T17:52:45.425ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
32602024-08-22T17:52:45.425ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
32612024-08-22T17:52:45.426ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32622024-08-22T17:52:45.427ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32632024-08-22T17:52:45.427ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
32642024-08-22T17:52:45.427ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
32652024-08-22T17:52:45.431ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
32662024-08-22T17:52:45.431ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
32672024-08-22T17:52:45.431ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
32682024-08-22T17:52:45.431ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
32692024-08-22T17:52:45.431ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
32702024-08-22T17:52:45.432ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
32712024-08-22T17:52:45.433ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32722024-08-22T17:52:45.433ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
32732024-08-22T17:52:45.433ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
32742024-08-22T17:52:45.437ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 147bdc10-344c-4ee8-a219-603cfc66e6d2, lookup_type: ById(147bdc10-344c-4ee8-a219-603cfc66e6d2) } result = Ok(())
32752024-08-22T17:52:45.437ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
32762024-08-22T17:52:45.437ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
32772024-08-22T17:52:45.440ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
32782024-08-22T17:52:45.440ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
32792024-08-22T17:52:45.444ZDEBGnexus (BackgroundTasks): client request SledAgent = c961ede4-7532-4ca1-95a5-690b4d7c9b5a background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
32802024-08-22T17:52:45.444ZDEBGnexus (BackgroundTasks): client response SledAgent = c961ede4-7532-4ca1-95a5-690b4d7c9b5a background_task = vpc_route_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/vpc-routes", query: None, fragment: None }, status: 204, headers: {"x-request-id": "db0f2ef5-29a6-432e-86fb-8eaed6ea66b9", "date": "Thu, 22 Aug 2024 17:52:44 GMT"} })
32812024-08-22T17:52:45.445ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 147bdc10-344c-4ee8-a219-603cfc66e6d2 original_time_created = 2024-08-22 17:49:18.445237 UTC target_id = 147bdc10-344c-4ee8-a219-603cfc66e6d2 time_created = 2024-08-22 17:49:18.445237 UTC
32822024-08-22T17:52:45.445ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 373.5552ms iteration = 19 status = {"status":"target blueprint unchanged","target_id":"147bdc10-344c-4ee8-a219-603cfc66e6d2","time_created":"2024-08-22T17:49:18.445237Z"}
32832024-08-22T17:52:45.446ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 373.883252ms iteration = 7 status = {}
32842024-08-22T17:52:45.457ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 384.818209ms iteration = 19 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
32852024-08-22T17:52:45.467ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
32862024-08-22T17:52:45.467ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 7f16ce95-fbde-4896-9a22-7f61d8d01c5d dns_zone_name = oxide.test found_so_far = 0 version = 2
32872024-08-22T17:52:45.467ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
32882024-08-22T17:52:45.467ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
32892024-08-22T17:52:45.469ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32902024-08-22T17:52:45.469ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32912024-08-22T17:52:45.471ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
32922024-08-22T17:52:45.471ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = d6f65b4e-2e87-480a-af21-a69b4520828c dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
32932024-08-22T17:52:45.471ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
32942024-08-22T17:52:45.471ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
32952024-08-22T17:52:45.472ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
32962024-08-22T17:52:45.472ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
32972024-08-22T17:52:45.472ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
32982024-08-22T17:52:45.472ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
32992024-08-22T17:52:45.472ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
33002024-08-22T17:52:45.472ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
33012024-08-22T17:52:45.472ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33022024-08-22T17:52:45.474ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
33032024-08-22T17:52:45.474ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
33042024-08-22T17:52:45.475ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
33052024-08-22T17:52:45.475ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
33062024-08-22T17:52:45.476ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
33072024-08-22T17:52:45.477ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33082024-08-22T17:52:45.478ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 7f16ce95-fbde-4896-9a22-7f61d8d01c5d dns_zone_name = oxide.test found_so_far = 1 version = 2
33092024-08-22T17:52:45.478ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
33102024-08-22T17:52:45.478ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-08-22 17:49:58.502032 UTC dns_group = external generation = 2
33112024-08-22T17:52:45.478ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 405.643197ms iteration = 4 status = {"generation":2}
33122024-08-22T17:52:45.478ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
33132024-08-22T17:52:45.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
33142024-08-22T17:52:45.479ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = d6f65b4e-2e87-480a-af21-a69b4520828c dns_zone_name = control-plane.oxide.internal found_so_far = 50 version = 1
33152024-08-22T17:52:45.479ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
33162024-08-22T17:52:45.479ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-08-22 17:49:57.683168 UTC dns_group = internal generation = 1
33172024-08-22T17:52:45.479ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 407.378415ms iteration = 4 status = {"generation":1}
33182024-08-22T17:52:45.479ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
33192024-08-22T17:52:45.479ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
33202024-08-22T17:52:45.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
33212024-08-22T17:52:45.481ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
33222024-08-22T17:52:45.481ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
33232024-08-22T17:52:45.483ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33242024-08-22T17:52:45.486ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: c961ede4-7532-4ca1-95a5-690b4d7c9b5a, lookup_type: ById(c961ede4-7532-4ca1-95a5-690b4d7c9b5a) } result = Ok(())
33252024-08-22T17:52:45.486ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
33262024-08-22T17:52:45.486ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
33272024-08-22T17:52:45.489ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 417.248357ms iteration = 7 status = 0
33282024-08-22T17:52:45.490ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1058 zone_address = fd00:1122:3344:101::2
33292024-08-22T17:52:45.490ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
33302024-08-22T17:52:45.491ZDEBGnexus (BackgroundTasks): client request background_task = crdb_node_id_collector body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/node/id
33312024-08-22T17:52:45.491ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "3fbfa5ad-b76a-4d83-a315-1f3f7ad64adc", "content-length": "26", "date": "Thu, 22 Aug 2024 17:52:44 GMT"} })
33322024-08-22T17:52:45.492ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1063 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
33332024-08-22T17:52:45.493ZDEBGnexus (BackgroundTasks): client response background_task = crdb_node_id_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::4)), port: Some(32222), path: "/node/id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "278b6ce2-2a17-47d1-b3ed-d32fa3bc7333", "content-length": "64", "date": "Thu, 22 Aug 2024 17:52:45 GMT"} })
33342024-08-22T17:52:45.493ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
33352024-08-22T17:52:45.494ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
33362024-08-22T17:52:45.494ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1095 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
33372024-08-22T17:52:45.497ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33382024-08-22T17:52:45.498ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
33392024-08-22T17:52:45.498ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
33402024-08-22T17:52:45.499ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
33412024-08-22T17:52:45.503ZDEBGnexus (BackgroundTasks): activation complete background_task = crdb_node_id_collector elapsed = 430.995908ms iteration = 3 status = {"nsuccess":5}
33422024-08-22T17:52:45.504ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:454 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
33432024-08-22T17:52:45.504ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 432.002703ms iteration = 7 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
33442024-08-22T17:52:45.527ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-08-22T17:49:43.060798Z, time_modified: 2024-08-22T17:49:43.060798Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 10d39b39-43d3-41db-b622-3ef3e09e2dad, db_silo: Silo { identity: SiloIdentity { id: 10d39b39-43d3-41db-b622-3ef3e09e2dad, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-08-22T17:49:58.442383Z, time_modified: 2024-08-22T17:49:58.442383Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "852177ddd7117113b778290a63910551c7b25a6718e9c8fda4590027c9443fee" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 10d39b39-43d3-41db-b622-3ef3e09e2dad, db_silo: Silo { identity: SiloIdentity { id: 10d39b39-43d3-41db-b622-3ef3e09e2dad, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-08-22T17:49:58.442383Z, time_modified: 2024-08-22T17:49:58.442383Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "852177ddd7117113b778290a63910551c7b25a6718e9c8fda4590027c9443fee" }] }) }
33452024-08-22T17:52:45.527ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 455.206012ms iteration = 4 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"10d39b39-43d3-41db-b622-3ef3e09e2dad","tls_certs":["852177ddd7117113b778290a63910551c7b25a6718e9c8fda4590027c9443fee"]}},"default_endpoint":{"silo_id":"10d39b39-43d3-41db-b622-3ef3e09e2dad","tls_certs":["852177ddd7117113b778290a63910551c7b25a6718e9c8fda4590027c9443fee"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
33462024-08-22T17:52:45.535ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
33472024-08-22T17:52:45.535ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
33482024-08-22T17:52:45.537ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33492024-08-22T17:52:45.538ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1058 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 zone_address = fd00:1122:3344:101::2
33502024-08-22T17:52:45.538ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
33512024-08-22T17:52:45.538ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "fd2973bf-432b-4169-a991-ebce0a7ec2ff", "content-length": "26", "date": "Thu, 22 Aug 2024 17:52:44 GMT"} })
33522024-08-22T17:52:45.538ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1063 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
33532024-08-22T17:52:45.539ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
33542024-08-22T17:52:45.539ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
33552024-08-22T17:52:45.539ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
33562024-08-22T17:52:45.541ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33572024-08-22T17:52:45.542ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1095 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561
33582024-08-22T17:52:45.543ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
33592024-08-22T17:52:45.543ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
33602024-08-22T17:52:45.543ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
33612024-08-22T17:52:45.570ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 498.611367ms iteration = 7 status = {}
33622024-08-22T17:52:45.572ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
33632024-08-22T17:52:45.581ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
33642024-08-22T17:52:45.582ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::9)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "036c6d7a-a4dd-4553-b69d-0a56318110ca", "date": "Thu, 22 Aug 2024 17:52:45 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
33652024-08-22T17:52:45.582ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
33662024-08-22T17:52:45.584ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::8)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "b74fda13-ba1d-42a6-ba3d-9a61ea58283a", "date": "Thu, 22 Aug 2024 17:52:45 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
33672024-08-22T17:52:45.584ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
33682024-08-22T17:52:45.586ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
33692024-08-22T17:52:45.586ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 513.886396ms iteration = 6 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
33702024-08-22T17:52:45.619ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
33712024-08-22T17:52:45.619ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "a6a280ba-08f8-4367-b9fa-6961cab6e1f1", "content-length": "1", "date": "Thu, 22 Aug 2024 17:52:45 GMT"} })
33722024-08-22T17:52:45.622ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
33732024-08-22T17:52:45.622ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
33742024-08-22T17:52:45.624ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 552.147693ms iteration = 7 status = null
33752024-08-22T17:52:45.671ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
33762024-08-22T17:52:45.671ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:2::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "48d53e0a-8a61-4204-b23e-2aae67cf8041", "date": "Thu, 22 Aug 2024 17:52:45 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
33772024-08-22T17:52:45.671ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
33782024-08-22T17:52:45.674ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:1::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "906b4045-9d1c-46e7-ad92-55be2c4d8a01", "date": "Thu, 22 Aug 2024 17:52:45 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
33792024-08-22T17:52:45.674ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
33802024-08-22T17:52:45.677ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:3::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "804231e4-0445-40b0-aac0-8627884000f3", "date": "Thu, 22 Aug 2024 17:52:45 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
33812024-08-22T17:52:45.677ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
33822024-08-22T17:52:45.679ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
33832024-08-22T17:52:45.679ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 607.374769ms iteration = 6 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
33842024-08-22T17:52:45.708ZDEBGnexus (BackgroundTasks): client request SledAgent = c961ede4-7532-4ca1-95a5-690b4d7c9b5a background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
33852024-08-22T17:52:45.708ZDEBGnexus (BackgroundTasks): client response SledAgent = c961ede4-7532-4ca1-95a5-690b4d7c9b5a background_task = v2p_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/v2p", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "ed460979-e4a9-4e28-9b1d-ab47c6fdd444", "content-length": "691", "date": "Thu, 22 Aug 2024 17:52:45 GMT"} })
33862024-08-22T17:52:45.708ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = buskin
33872024-08-22T17:52:45.708ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = buskin
33882024-08-22T17:52:45.708ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = buskin
33892024-08-22T17:52:45.710ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 637.978807ms iteration = 7 status = {}
33902024-08-22T17:52:45.752ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
33912024-08-22T17:52:45.752ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
33922024-08-22T17:52:45.755ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:147 port = SwitchPort { id: c772d372-7448-4ce0-b8d1-efa645d18ed1, rack_id: c66d93a6-e2c3-4b19-9564-3b58bad50561, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(9ff05247-84aa-47d8-8619-ce0a8dbbaf7c) } rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 switch_location = Switch0
33932024-08-22T17:52:45.755ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
33942024-08-22T17:52:45.755ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
33952024-08-22T17:52:45.766ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
33962024-08-22T17:52:45.766ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/port/qsfp0/settings", query: Some("tag=omicron"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "692214bc-bf05-438b-a89c-0e873ae6f109", "content-length": "121", "date": "Thu, 22 Aug 2024 17:52:45 GMT"} })
33972024-08-22T17:52:45.766ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1714 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G } }} } switch_port_id = "qsfp0"
33982024-08-22T17:52:45.766ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1741 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 9ff05247-84aa-47d8-8619-ce0a8dbbaf7c
33992024-08-22T17:52:45.766ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:379 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
34002024-08-22T17:52:45.766ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
34012024-08-22T17:52:45.766ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:392 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 routes = {}
34022024-08-22T17:52:45.766ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:398 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 routes = {}
34032024-08-22T17:52:45.766ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:418 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561
34042024-08-22T17:52:45.766ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
34052024-08-22T17:52:45.767ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
34062024-08-22T17:52:45.769ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
34072024-08-22T17:52:45.769ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv4", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "24eb0c8e-5a98-4eb3-a11c-3b6bc4bbe1da", "content-length": "2", "date": "Thu, 22 Aug 2024 17:52:45 GMT"} })
34082024-08-22T17:52:45.769ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
34092024-08-22T17:52:45.769ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "e35ef450-c2fd-47d6-b606-12a6697537d2", "content-length": "52", "date": "Thu, 22 Aug 2024 17:52:45 GMT"} })
34102024-08-22T17:52:45.769ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0" }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:467 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 switch_location = Switch0
34112024-08-22T17:52:45.769ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
34122024-08-22T17:52:45.866ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/switch-ports", query: None, fragment: None }, status: 204, headers: {"x-request-id": "c9ac5a07-e599-4833-95b3-5c1247cd7e4d", "date": "Thu, 22 Aug 2024 17:52:45 GMT"} })
34132024-08-22T17:52:45.866ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:770 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 switch = Switch0
34142024-08-22T17:52:45.866ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
34152024-08-22T17:52:45.867ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/network-bootstore-config", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "a4197b62-c686-4425-9015-65bf961c4e82", "content-length": "667", "date": "Thu, 22 Aug 2024 17:52:45 GMT"} })
34162024-08-22T17:52:45.867ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
34172024-08-22T17:52:45.867ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
34182024-08-22T17:52:45.871ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
34192024-08-22T17:52:45.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
34202024-08-22T17:52:45.875ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
34212024-08-22T17:52:45.875ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
34222024-08-22T17:52:45.877ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
34232024-08-22T17:52:45.878ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
34242024-08-22T17:52:45.884ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 812.209318ms iteration = 7 status = {}
34252024-08-22T17:52:45.914ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 190 local_addr = [fd00:1122:3344:101::c]:56994 method = GET remote_addr = [fd00:1122:3344:101::d]:40670 req_id = 202d8d4f-e80e-4864-b669-66abbb228772 response_code = 200 uri = /eaa260db-2b19-4ae1-97e8-5735c9f5d955
34262024-08-22T17:52:55.071ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 20 reason = Timeout
34272024-08-22T17:52:55.072ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 20 reason = Timeout
34282024-08-22T17:52:55.072ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
34292024-08-22T17:52:55.072ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
34302024-08-22T17:52:55.072ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
34312024-08-22T17:52:55.072ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
34322024-08-22T17:52:55.073ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34332024-08-22T17:52:55.074ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
34342024-08-22T17:52:55.074ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
34352024-08-22T17:52:55.076ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
34362024-08-22T17:52:55.076ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
34372024-08-22T17:52:55.076ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
34382024-08-22T17:52:55.078ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
34392024-08-22T17:52:55.078ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
34402024-08-22T17:52:55.079ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34412024-08-22T17:52:55.081ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 147bdc10-344c-4ee8-a219-603cfc66e6d2, lookup_type: ById(147bdc10-344c-4ee8-a219-603cfc66e6d2) } result = Ok(())
34422024-08-22T17:52:55.081ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
34432024-08-22T17:52:55.082ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
34442024-08-22T17:52:55.092ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 147bdc10-344c-4ee8-a219-603cfc66e6d2 original_time_created = 2024-08-22 17:49:18.445237 UTC target_id = 147bdc10-344c-4ee8-a219-603cfc66e6d2 time_created = 2024-08-22 17:49:18.445237 UTC
34452024-08-22T17:52:55.092ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 20.326735ms iteration = 20 status = {"status":"target blueprint unchanged","target_id":"147bdc10-344c-4ee8-a219-603cfc66e6d2","time_created":"2024-08-22T17:49:18.445237Z"}
34462024-08-22T17:52:55.105ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 33.248692ms iteration = 20 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
34472024-08-22T17:52:55.914ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 232 local_addr = [fd00:1122:3344:101::c]:56994 method = GET remote_addr = [fd00:1122:3344:101::d]:40670 req_id = da4c4384-45c7-4897-a520-9e6eebb2ff62 response_code = 200 uri = /eaa260db-2b19-4ae1-97e8-5735c9f5d955
34482024-08-22T17:53:05.072ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 21 reason = Timeout
34492024-08-22T17:53:05.072ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 21 reason = Timeout
34502024-08-22T17:53:05.072ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
34512024-08-22T17:53:05.072ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
34522024-08-22T17:53:05.072ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
34532024-08-22T17:53:05.072ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
34542024-08-22T17:53:05.073ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34552024-08-22T17:53:05.074ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
34562024-08-22T17:53:05.075ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
34572024-08-22T17:53:05.076ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
34582024-08-22T17:53:05.076ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
34592024-08-22T17:53:05.076ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
34602024-08-22T17:53:05.078ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
34612024-08-22T17:53:05.078ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
34622024-08-22T17:53:05.080ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34632024-08-22T17:53:05.082ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 147bdc10-344c-4ee8-a219-603cfc66e6d2, lookup_type: ById(147bdc10-344c-4ee8-a219-603cfc66e6d2) } result = Ok(())
34642024-08-22T17:53:05.082ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
34652024-08-22T17:53:05.082ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
34662024-08-22T17:53:05.088ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 147bdc10-344c-4ee8-a219-603cfc66e6d2 original_time_created = 2024-08-22 17:49:18.445237 UTC target_id = 147bdc10-344c-4ee8-a219-603cfc66e6d2 time_created = 2024-08-22 17:49:18.445237 UTC
34672024-08-22T17:53:05.088ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 16.437536ms iteration = 21 status = {"status":"target blueprint unchanged","target_id":"147bdc10-344c-4ee8-a219-603cfc66e6d2","time_created":"2024-08-22T17:49:18.445237Z"}
34682024-08-22T17:53:05.105ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 33.547793ms iteration = 21 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
34692024-08-22T17:53:05.915ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 1271 local_addr = [fd00:1122:3344:101::c]:56994 method = GET remote_addr = [fd00:1122:3344:101::d]:40670 req_id = 2cfef386-dd24-4926-b996-5ec142c96266 response_code = 200 uri = /eaa260db-2b19-4ae1-97e8-5735c9f5d955
34702024-08-22T17:53:15.072ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 22 reason = Timeout
34712024-08-22T17:53:15.072ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 8 reason = Timeout
34722024-08-22T17:53:15.072ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
34732024-08-22T17:53:15.072ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
34742024-08-22T17:53:15.072ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 8 reason = Timeout
34752024-08-22T17:53:15.072ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
34762024-08-22T17:53:15.072ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 8 reason = Timeout
34772024-08-22T17:53:15.072ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
34782024-08-22T17:53:15.072ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 8 reason = Timeout
34792024-08-22T17:53:15.072ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 8 reason = Timeout
34802024-08-22T17:53:15.072ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
34812024-08-22T17:53:15.072ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 8 reason = Timeout
34822024-08-22T17:53:15.072ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
34832024-08-22T17:53:15.072ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
34842024-08-22T17:53:15.072ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 8 reason = Timeout
34852024-08-22T17:53:15.072ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
34862024-08-22T17:53:15.072ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 8 reason = Timeout
34872024-08-22T17:53:15.072ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
34882024-08-22T17:53:15.072ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
34892024-08-22T17:53:15.072ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 8 reason = Timeout
34902024-08-22T17:53:15.072ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
34912024-08-22T17:53:15.072ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 22 reason = Timeout
34922024-08-22T17:53:15.072ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
34932024-08-22T17:53:15.072ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 8 reason = Timeout
34942024-08-22T17:53:15.072ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
34952024-08-22T17:53:15.072ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 8 reason = Timeout
34962024-08-22T17:53:15.072ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 9 reason = Timeout
34972024-08-22T17:53:15.072ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
34982024-08-22T17:53:15.072ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
34992024-08-22T17:53:15.072ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
35002024-08-22T17:53:15.072ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
35012024-08-22T17:53:15.072ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
35022024-08-22T17:53:15.072ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
35032024-08-22T17:53:15.072ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
35042024-08-22T17:53:15.072ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
35052024-08-22T17:53:15.072ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
35062024-08-22T17:53:15.072ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
35072024-08-22T17:53:15.072ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
35082024-08-22T17:53:15.072ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
35092024-08-22T17:53:15.072ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
35102024-08-22T17:53:15.072ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
35112024-08-22T17:53:15.074ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35122024-08-22T17:53:15.074ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 8 reason = Timeout
35132024-08-22T17:53:15.074ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
35142024-08-22T17:53:15.074ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
35152024-08-22T17:53:15.075ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35162024-08-22T17:53:15.076ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.97852ms iteration = 8 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
35172024-08-22T17:53:15.076ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-c961ede4-7532-4ca1-95a5-690b4d7c9b5a.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-c961ede4-7532-4ca1-95a5-690b4d7c9b5a.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1783, tv_nsec: 605767727 } })
35182024-08-22T17:53:15.077ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1050
35192024-08-22T17:53:15.077ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
35202024-08-22T17:53:15.077ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
35212024-08-22T17:53:15.077ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
35222024-08-22T17:53:15.077ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
35232024-08-22T17:53:15.077ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
35242024-08-22T17:53:15.077ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
35252024-08-22T17:53:15.079ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35262024-08-22T17:53:15.081ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:55
35272024-08-22T17:53:15.081ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
35282024-08-22T17:53:15.081ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
35292024-08-22T17:53:15.081ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
35302024-08-22T17:53:15.081ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-c961ede4-7532-4ca1-95a5-690b4d7c9b5a.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-c961ede4-7532-4ca1-95a5-690b4d7c9b5a.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1783, tv_nsec: 610505071 } })
35312024-08-22T17:53:15.081ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1050
35322024-08-22T17:53:15.082ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
35332024-08-22T17:53:15.082ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
35342024-08-22T17:53:15.082ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
35352024-08-22T17:53:15.083ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35362024-08-22T17:53:15.084ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35372024-08-22T17:53:15.085ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
35382024-08-22T17:53:15.086ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
35392024-08-22T17:53:15.086ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
35402024-08-22T17:53:15.086ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
35412024-08-22T17:53:15.086ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
35422024-08-22T17:53:15.086ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
35432024-08-22T17:53:15.086ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
35442024-08-22T17:53:15.086ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
35452024-08-22T17:53:15.089ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-c961ede4-7532-4ca1-95a5-690b4d7c9b5a.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-c961ede4-7532-4ca1-95a5-690b4d7c9b5a.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1783, tv_nsec: 618004820 } })
35462024-08-22T17:53:15.091ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1050 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561
35472024-08-22T17:53:15.092ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35482024-08-22T17:53:15.096ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 147bdc10-344c-4ee8-a219-603cfc66e6d2, lookup_type: ById(147bdc10-344c-4ee8-a219-603cfc66e6d2) } result = Ok(())
35492024-08-22T17:53:15.096ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
35502024-08-22T17:53:15.096ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
35512024-08-22T17:53:15.099ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:454 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
35522024-08-22T17:53:15.099ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 27.563762ms iteration = 8 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
35532024-08-22T17:53:15.100ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
35542024-08-22T17:53:15.100ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
35552024-08-22T17:53:15.102ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35562024-08-22T17:53:15.103ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 147bdc10-344c-4ee8-a219-603cfc66e6d2 original_time_created = 2024-08-22 17:49:18.445237 UTC target_id = 147bdc10-344c-4ee8-a219-603cfc66e6d2 time_created = 2024-08-22 17:49:18.445237 UTC
35572024-08-22T17:53:15.103ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 31.912385ms iteration = 22 status = {"status":"target blueprint unchanged","target_id":"147bdc10-344c-4ee8-a219-603cfc66e6d2","time_created":"2024-08-22T17:49:18.445237Z"}
35582024-08-22T17:53:15.104ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
35592024-08-22T17:53:15.104ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
35602024-08-22T17:53:15.104ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
35612024-08-22T17:53:15.122ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35622024-08-22T17:53:15.126ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
35632024-08-22T17:53:15.126ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
35642024-08-22T17:53:15.126ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
35652024-08-22T17:53:15.126ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
35662024-08-22T17:53:15.126ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
35672024-08-22T17:53:15.126ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
35682024-08-22T17:53:15.127ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
35692024-08-22T17:53:15.127ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
35702024-08-22T17:53:15.127ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
35712024-08-22T17:53:15.127ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:55
35722024-08-22T17:53:15.127ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:220 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
35732024-08-22T17:53:15.127ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 55.468827ms iteration = 8 status = {"destroyed_active_vmms":0,"error":null,"saga_start_failures":0,"sagas_completed":0,"sagas_failed":0,"sagas_started":0,"terminated_active_migrations":0}
35742024-08-22T17:53:15.129ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
35752024-08-22T17:53:15.129ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 56.974045ms iteration = 22 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
35762024-08-22T17:53:15.129ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
35772024-08-22T17:53:15.132ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 60.196541ms iteration = 8 status = {"region_replacement_started_err":0,"region_replacement_started_ok":0}
35782024-08-22T17:53:15.132ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 58.299791ms iteration = 8 status = {"errors":[],"garbage_collect_requested":[]}
35792024-08-22T17:53:15.134ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35802024-08-22T17:53:15.135ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
35812024-08-22T17:53:15.135ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
35822024-08-22T17:53:15.135ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
35832024-08-22T17:53:15.159ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
35842024-08-22T17:53:15.160ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
35852024-08-22T17:53:15.162ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
35862024-08-22T17:53:15.163ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
35872024-08-22T17:53:15.164ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35882024-08-22T17:53:15.168ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 95.981696ms iteration = 8 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
35892024-08-22T17:53:15.168ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: c961ede4-7532-4ca1-95a5-690b4d7c9b5a, lookup_type: ById(c961ede4-7532-4ca1-95a5-690b4d7c9b5a) } result = Ok(())
35902024-08-22T17:53:15.168ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
35912024-08-22T17:53:15.168ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
35922024-08-22T17:53:15.171ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 98.935961ms iteration = 8 status = 0
35932024-08-22T17:53:15.199ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1058 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 zone_address = fd00:1122:3344:101::2
35942024-08-22T17:53:15.199ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
35952024-08-22T17:53:15.200ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "7b5c14ed-c1a8-4d6d-969d-84e539dae030", "content-length": "26", "date": "Thu, 22 Aug 2024 17:53:14 GMT"} })
35962024-08-22T17:53:15.200ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1063 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
35972024-08-22T17:53:15.201ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1095 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561
35982024-08-22T17:53:15.208ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 136.520175ms iteration = 9 status = {"physical_disks_added":0}
35992024-08-22T17:53:15.210ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
36002024-08-22T17:53:15.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
36012024-08-22T17:53:15.241ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
36022024-08-22T17:53:15.241ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
36032024-08-22T17:53:15.268ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1058 zone_address = fd00:1122:3344:101::2
36042024-08-22T17:53:15.268ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
36052024-08-22T17:53:15.268ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "e6ff8484-f622-415a-9cd8-cb9e58e62f7b", "content-length": "26", "date": "Thu, 22 Aug 2024 17:53:14 GMT"} })
36062024-08-22T17:53:15.268ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1063 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
36072024-08-22T17:53:15.270ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1095 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
36082024-08-22T17:53:15.297ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1058 zone_address = fd00:1122:3344:101::2
36092024-08-22T17:53:15.297ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
36102024-08-22T17:53:15.297ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "b0ed91ef-2976-47a3-afcb-1ff45f32c730", "content-length": "26", "date": "Thu, 22 Aug 2024 17:53:14 GMT"} })
36112024-08-22T17:53:15.298ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1063 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
36122024-08-22T17:53:15.299ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1095 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
36132024-08-22T17:53:15.311ZDEBGnexus (BackgroundTasks): client request SledAgent = c961ede4-7532-4ca1-95a5-690b4d7c9b5a background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
36142024-08-22T17:53:15.311ZDEBGnexus (BackgroundTasks): client response SledAgent = c961ede4-7532-4ca1-95a5-690b4d7c9b5a background_task = vpc_route_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/vpc-routes", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "c300fe50-1aa4-4909-bc7b-d04cf7407f8a", "content-length": "372", "date": "Thu, 22 Aug 2024 17:53:14 GMT"} })
36152024-08-22T17:53:15.311ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
36162024-08-22T17:53:15.312ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
36172024-08-22T17:53:15.313ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36182024-08-22T17:53:15.314ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
36192024-08-22T17:53:15.314ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
36202024-08-22T17:53:15.314ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
36212024-08-22T17:53:15.314ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
36222024-08-22T17:53:15.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
36232024-08-22T17:53:15.316ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36242024-08-22T17:53:15.317ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
36252024-08-22T17:53:15.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
36262024-08-22T17:53:15.318ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
36272024-08-22T17:53:15.318ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
36282024-08-22T17:53:15.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
36292024-08-22T17:53:15.322ZDEBGnexus (BackgroundTasks): client request SledAgent = c961ede4-7532-4ca1-95a5-690b4d7c9b5a background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
36302024-08-22T17:53:15.322ZDEBGnexus (BackgroundTasks): client response SledAgent = c961ede4-7532-4ca1-95a5-690b4d7c9b5a background_task = vpc_route_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/vpc-routes", query: None, fragment: None }, status: 204, headers: {"x-request-id": "a81c81b5-79ea-4b59-ae37-54eb672e44c1", "date": "Thu, 22 Aug 2024 17:53:14 GMT"} })
36312024-08-22T17:53:15.324ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 252.566465ms iteration = 8 status = {}
36322024-08-22T17:53:15.358ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
36332024-08-22T17:53:15.359ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "3875f691-6c21-4f9b-b64b-73d4f82d8c61", "content-length": "1", "date": "Thu, 22 Aug 2024 17:53:14 GMT"} })
36342024-08-22T17:53:15.360ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
36352024-08-22T17:53:15.360ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
36362024-08-22T17:53:15.362ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 290.50553ms iteration = 8 status = null
36372024-08-22T17:53:15.380ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 308.322564ms iteration = 8 status = {}
36382024-08-22T17:53:15.439ZDEBGnexus (BackgroundTasks): client request SledAgent = c961ede4-7532-4ca1-95a5-690b4d7c9b5a background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
36392024-08-22T17:53:15.439ZDEBGnexus (BackgroundTasks): client response SledAgent = c961ede4-7532-4ca1-95a5-690b4d7c9b5a background_task = v2p_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/v2p", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "a298f8b6-29c9-4954-9327-458f9676c33f", "content-length": "691", "date": "Thu, 22 Aug 2024 17:53:14 GMT"} })
36402024-08-22T17:53:15.439ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = buskin
36412024-08-22T17:53:15.439ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = buskin
36422024-08-22T17:53:15.439ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = buskin
36432024-08-22T17:53:15.440ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 368.480554ms iteration = 8 status = {}
36442024-08-22T17:53:15.443ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
36452024-08-22T17:53:15.443ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
36462024-08-22T17:53:15.445ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:147 port = SwitchPort { id: c772d372-7448-4ce0-b8d1-efa645d18ed1, rack_id: c66d93a6-e2c3-4b19-9564-3b58bad50561, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(9ff05247-84aa-47d8-8619-ce0a8dbbaf7c) } rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 switch_location = Switch0
36472024-08-22T17:53:15.445ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
36482024-08-22T17:53:15.445ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
36492024-08-22T17:53:15.555ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
36502024-08-22T17:53:15.556ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/port/qsfp0/settings", query: Some("tag=omicron"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "ed013b8d-01f2-4136-98d3-9d12461309be", "content-length": "121", "date": "Thu, 22 Aug 2024 17:53:14 GMT"} })
36512024-08-22T17:53:15.556ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1714 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G } }} } switch_port_id = "qsfp0"
36522024-08-22T17:53:15.556ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1741 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 9ff05247-84aa-47d8-8619-ce0a8dbbaf7c
36532024-08-22T17:53:15.556ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:379 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
36542024-08-22T17:53:15.556ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
36552024-08-22T17:53:15.556ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:392 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 routes = {}
36562024-08-22T17:53:15.556ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:398 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 routes = {}
36572024-08-22T17:53:15.556ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:418 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561
36582024-08-22T17:53:15.556ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
36592024-08-22T17:53:15.557ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
36602024-08-22T17:53:15.558ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
36612024-08-22T17:53:15.558ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv4", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "3f7fd0c5-874a-4ab5-97a1-8b0e41f5ecbe", "content-length": "2", "date": "Thu, 22 Aug 2024 17:53:14 GMT"} })
36622024-08-22T17:53:15.558ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
36632024-08-22T17:53:15.558ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "7623fc5e-15a7-4d9e-a6f6-98b169fe9ad4", "content-length": "52", "date": "Thu, 22 Aug 2024 17:53:14 GMT"} })
36642024-08-22T17:53:15.558ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0" }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:467 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 switch_location = Switch0
36652024-08-22T17:53:15.558ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
36662024-08-22T17:53:15.672ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/switch-ports", query: None, fragment: None }, status: 204, headers: {"x-request-id": "0771b193-78a1-42b7-8043-9caffe3ec07e", "date": "Thu, 22 Aug 2024 17:53:15 GMT"} })
36672024-08-22T17:53:15.673ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:770 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 switch = Switch0
36682024-08-22T17:53:15.673ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
36692024-08-22T17:53:15.673ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/network-bootstore-config", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "aaeed92a-e55c-4bcc-9cce-f15344b54e61", "content-length": "667", "date": "Thu, 22 Aug 2024 17:53:15 GMT"} })
36702024-08-22T17:53:15.673ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
36712024-08-22T17:53:15.673ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
36722024-08-22T17:53:15.677ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
36732024-08-22T17:53:15.678ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
36742024-08-22T17:53:15.682ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
36752024-08-22T17:53:15.682ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
36762024-08-22T17:53:15.684ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
36772024-08-22T17:53:15.684ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
36782024-08-22T17:53:15.693ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 621.156561ms iteration = 8 status = {}
36792024-08-22T17:53:15.888ZDEBGnexus (producer-registration-task): registering / renewing oximeter producer lease with Nexus
36802024-08-22T17:53:15.888ZDEBGnexus (producer-registration-task): using nexus address for registration addr = [fd00:1122:3344:101::c]:12221
36812024-08-22T17:53:15.919ZDEBGnexus (producer-registration-task): client request body = Some(Body) method = POST uri = http://[fd00:1122:3344:101::c]:12221/metrics/producers
36822024-08-22T17:53:15.919ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::c]:51317
36832024-08-22T17:53:15.920ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:51317 req_id = a1e4c082-5e02-4b65-be82-7bf472694ae0 roles = RoleSet { roles: {} } uri = /metrics/producers
36842024-08-22T17:53:15.920ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 809 local_addr = [fd00:1122:3344:101::c]:56994 method = GET remote_addr = [fd00:1122:3344:101::d]:40670 req_id = d5a0ecb8-6943-4566-a763-875fef98ad1a response_code = 200 uri = /eaa260db-2b19-4ae1-97e8-5735c9f5d955
36852024-08-22T17:53:15.920ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:51317 req_id = a1e4c082-5e02-4b65-be82-7bf472694ae0 resource = Database result = Ok(()) uri = /metrics/producers
36862024-08-22T17:53:15.964ZINFOnexus: registered oximeter collector client file = nexus/src/app/oximeter.rs:338 id = 7fc6f696-ffb5-46eb-ae34-da7098d2b35a
36872024-08-22T17:53:15.964ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:51317 req_id = a1e4c082-5e02-4b65-be82-7bf472694ae0 roles = RoleSet { roles: {} } uri = /metrics/producers
36882024-08-22T17:53:15.964ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:51317 req_id = a1e4c082-5e02-4b65-be82-7bf472694ae0 resource = Database result = Ok(()) uri = /metrics/producers
36892024-08-22T17:53:16.004ZDEBGnexus: client request body = Some(Body) method = POST oximeter-collector = 7fc6f696-ffb5-46eb-ae34-da7098d2b35a uri = http://[fd00:1122:3344:101::d]:12223/producers
36902024-08-22T17:53:16.004ZDEBGnexus: client response oximeter-collector = 7fc6f696-ffb5-46eb-ae34-da7098d2b35a result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::d)), port: Some(12223), path: "/producers", query: None, fragment: None }, status: 204, headers: {"x-request-id": "0b9b1825-735e-4ba5-bb76-0384059e65d6", "date": "Thu, 22 Aug 2024 17:53:16 GMT"} })
36912024-08-22T17:53:16.004ZINFOnexus: assigned collector to new producer collector_id = 7fc6f696-ffb5-46eb-ae34-da7098d2b35a file = nexus/src/app/oximeter.rs:125 producer_id = eaa260db-2b19-4ae1-97e8-5735c9f5d955
36922024-08-22T17:53:16.005ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 86443 local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:51317 req_id = a1e4c082-5e02-4b65-be82-7bf472694ae0 response_code = 201 uri = /metrics/producers
36932024-08-22T17:53:16.006ZDEBGnexus (producer-registration-task): client response result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::c)), port: Some(12221), path: "/metrics/producers", query: None, fragment: None }, status: 201, headers: {"content-type": "application/json", "x-request-id": "a1e4c082-5e02-4b65-be82-7bf472694ae0", "content-length": "41", "date": "Thu, 22 Aug 2024 17:53:15 GMT"} })
36942024-08-22T17:53:16.007ZDEBGnexus (producer-registration-task): registered with nexus successfully
36952024-08-22T17:53:16.007ZDEBGnexus (producer-registration-task): pausing until time to renew lease lease_duration = 600s wait_period = 150s
36962024-08-22T17:53:25.072ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 23 reason = Timeout
36972024-08-22T17:53:25.072ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 23 reason = Timeout
36982024-08-22T17:53:25.072ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
36992024-08-22T17:53:25.072ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
37002024-08-22T17:53:25.072ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
37012024-08-22T17:53:25.072ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
37022024-08-22T17:53:25.074ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37032024-08-22T17:53:25.074ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
37042024-08-22T17:53:25.074ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
37052024-08-22T17:53:25.076ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.035091ms iteration = 23 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
37062024-08-22T17:53:25.076ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
37072024-08-22T17:53:25.076ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
37082024-08-22T17:53:25.076ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
37092024-08-22T17:53:25.078ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
37102024-08-22T17:53:25.078ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
37112024-08-22T17:53:25.104ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37122024-08-22T17:53:25.107ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 147bdc10-344c-4ee8-a219-603cfc66e6d2, lookup_type: ById(147bdc10-344c-4ee8-a219-603cfc66e6d2) } result = Ok(())
37132024-08-22T17:53:25.107ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
37142024-08-22T17:53:25.107ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
37152024-08-22T17:53:25.112ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 147bdc10-344c-4ee8-a219-603cfc66e6d2 original_time_created = 2024-08-22 17:49:18.445237 UTC target_id = 147bdc10-344c-4ee8-a219-603cfc66e6d2 time_created = 2024-08-22 17:49:18.445237 UTC
37162024-08-22T17:53:25.112ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 40.48115ms iteration = 23 status = {"status":"target blueprint unchanged","target_id":"147bdc10-344c-4ee8-a219-603cfc66e6d2","time_created":"2024-08-22T17:49:18.445237Z"}
37172024-08-22T17:53:26.005ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 242 local_addr = [fd00:1122:3344:101::c]:56994 method = GET remote_addr = [fd00:1122:3344:101::d]:40670 req_id = 6f4b32d4-5215-4b83-91d6-49faddb084ce response_code = 200 uri = /eaa260db-2b19-4ae1-97e8-5735c9f5d955
37182024-08-22T17:53:35.073ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 24 reason = Timeout
37192024-08-22T17:53:35.073ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 24 reason = Timeout
37202024-08-22T17:53:35.073ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
37212024-08-22T17:53:35.073ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
37222024-08-22T17:53:35.073ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
37232024-08-22T17:53:35.073ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
37242024-08-22T17:53:35.074ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37252024-08-22T17:53:35.075ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
37262024-08-22T17:53:35.076ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
37272024-08-22T17:53:35.077ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
37282024-08-22T17:53:35.077ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
37292024-08-22T17:53:35.077ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
37302024-08-22T17:53:35.079ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
37312024-08-22T17:53:35.079ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
37322024-08-22T17:53:35.080ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37332024-08-22T17:53:35.083ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 147bdc10-344c-4ee8-a219-603cfc66e6d2, lookup_type: ById(147bdc10-344c-4ee8-a219-603cfc66e6d2) } result = Ok(())
37342024-08-22T17:53:35.083ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
37352024-08-22T17:53:35.083ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
37362024-08-22T17:53:35.091ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 147bdc10-344c-4ee8-a219-603cfc66e6d2 original_time_created = 2024-08-22 17:49:18.445237 UTC target_id = 147bdc10-344c-4ee8-a219-603cfc66e6d2 time_created = 2024-08-22 17:49:18.445237 UTC
37372024-08-22T17:53:35.091ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 18.180985ms iteration = 24 status = {"status":"target blueprint unchanged","target_id":"147bdc10-344c-4ee8-a219-603cfc66e6d2","time_created":"2024-08-22T17:49:18.445237Z"}
37382024-08-22T17:53:35.108ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 35.576824ms iteration = 24 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
37392024-08-22T17:53:36.006ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 217 local_addr = [fd00:1122:3344:101::c]:56994 method = GET remote_addr = [fd00:1122:3344:101::d]:40670 req_id = 6d13adbe-2a5b-4d4e-9c85-e49776500201 response_code = 200 uri = /eaa260db-2b19-4ae1-97e8-5735c9f5d955
37402024-08-22T17:53:45.072ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 5 reason = Timeout
37412024-08-22T17:53:45.072ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 5 reason = Timeout
37422024-08-22T17:53:45.072ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
37432024-08-22T17:53:45.072ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 7 reason = Timeout
37442024-08-22T17:53:45.072ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 9 reason = Timeout
37452024-08-22T17:53:45.072ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 9 reason = Timeout
37462024-08-22T17:53:45.072ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 9 reason = Timeout
37472024-08-22T17:53:45.072ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
37482024-08-22T17:53:45.072ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
37492024-08-22T17:53:45.072ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
37502024-08-22T17:53:45.072ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
37512024-08-22T17:53:45.072ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 9 reason = Timeout
37522024-08-22T17:53:45.072ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 5 reason = Timeout
37532024-08-22T17:53:45.072ZINFOnexus (BackgroundTasks): Abandoned VMM reaper running background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:76
37542024-08-22T17:53:45.072ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
37552024-08-22T17:53:45.072ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
37562024-08-22T17:53:45.072ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 9 reason = Timeout
37572024-08-22T17:53:45.072ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 25 reason = Timeout
37582024-08-22T17:53:45.072ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
37592024-08-22T17:53:45.072ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
37602024-08-22T17:53:45.073ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 5 reason = Timeout
37612024-08-22T17:53:45.073ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
37622024-08-22T17:53:45.073ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
37632024-08-22T17:53:45.073ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 9 reason = Timeout
37642024-08-22T17:53:45.073ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
37652024-08-22T17:53:45.073ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 9 reason = Timeout
37662024-08-22T17:53:45.073ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
37672024-08-22T17:53:45.073ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 5 reason = Timeout
37682024-08-22T17:53:45.073ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:56
37692024-08-22T17:53:45.073ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
37702024-08-22T17:53:45.073ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 9 reason = Timeout
37712024-08-22T17:53:45.073ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 9 reason = Timeout
37722024-08-22T17:53:45.073ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
37732024-08-22T17:53:45.073ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 10 reason = Timeout
37742024-08-22T17:53:45.073ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 25 reason = Timeout
37752024-08-22T17:53:45.073ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
37762024-08-22T17:53:45.073ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
37772024-08-22T17:53:45.073ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 6 reason = Timeout
37782024-08-22T17:53:45.073ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:80 reason = blueprint disabled target_id = 147bdc10-344c-4ee8-a219-603cfc66e6d2
37792024-08-22T17:53:45.073ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 5 reason = Timeout
37802024-08-22T17:53:45.073ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 7 reason = Timeout
37812024-08-22T17:53:45.073ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2024-08-22 17:43:45.072988520 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
37822024-08-22T17:53:45.073ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 10.96\u{b5}s iteration = 6 status = {"error":"blueprint disabled","target_id":"147bdc10-344c-4ee8-a219-603cfc66e6d2"}
37832024-08-22T17:53:45.073ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
37842024-08-22T17:53:45.073ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 5 reason = Timeout
37852024-08-22T17:53:45.073ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 9 reason = Timeout
37862024-08-22T17:53:45.073ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 9 reason = Timeout
37872024-08-22T17:53:45.073ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
37882024-08-22T17:53:45.073ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
37892024-08-22T17:53:45.073ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 5 reason = Timeout
37902024-08-22T17:53:45.073ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
37912024-08-22T17:53:45.073ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 5 reason = Timeout
37922024-08-22T17:53:45.073ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
37932024-08-22T17:53:45.073ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
37942024-08-22T17:53:45.073ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
37952024-08-22T17:53:45.073ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
37962024-08-22T17:53:45.073ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
37972024-08-22T17:53:45.073ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
37982024-08-22T17:53:45.073ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
37992024-08-22T17:53:45.073ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
38002024-08-22T17:53:45.073ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
38012024-08-22T17:53:45.073ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
38022024-08-22T17:53:45.073ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
38032024-08-22T17:53:45.073ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
38042024-08-22T17:53:45.073ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
38052024-08-22T17:53:45.073ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
38062024-08-22T17:53:45.073ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
38072024-08-22T17:53:45.073ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
38082024-08-22T17:53:45.073ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
38092024-08-22T17:53:45.073ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("9488ce0a-6732-4f75-baf9-4ecfa406df91.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("afac2eab-cadf-48d0-94a7-ad562ade88c6.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("afac2eab-cadf-48d0-94a7-ad562ade88c6.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::9))) }], valid_until: Instant { tv_sec: 1813, tv_nsec: 601831668 } })
38102024-08-22T17:53:45.073ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
38112024-08-22T17:53:45.073ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
38122024-08-22T17:53:45.073ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
38132024-08-22T17:53:45.073ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
38142024-08-22T17:53:45.073ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
38152024-08-22T17:53:45.073ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("6507c79b-9a18-46c7-930d-c98e8662b87f.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("a09ea923-b85c-428d-b289-6936a0c2542c.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("c0ef020c-52c1-4b9a-a085-445fab31264f.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("c0ef020c-52c1-4b9a-a085-445fab31264f.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:3::1))) }], valid_until: Instant { tv_sec: 1813, tv_nsec: 602052200 } })
38162024-08-22T17:53:45.074ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 9 reason = Timeout
38172024-08-22T17:53:45.074ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
38182024-08-22T17:53:45.074ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
38192024-08-22T17:53:45.075ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38202024-08-22T17:53:45.077ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
38212024-08-22T17:53:45.077ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 4.571044ms iteration = 5 status = {"addresses":["[fd00:1122:3344:101::8]:5353","[fd00:1122:3344:101::9]:5353"]}
38222024-08-22T17:53:45.077ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38232024-08-22T17:53:45.077ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 4.791975ms iteration = 5 status = {"errors":[],"found_port_ok":[]}
38242024-08-22T17:53:45.077ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 4.864995ms iteration = 9 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
38252024-08-22T17:53:45.077ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
38262024-08-22T17:53:45.077ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
38272024-08-22T17:53:45.078ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
38282024-08-22T17:53:45.079ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
38292024-08-22T17:53:45.079ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
38302024-08-22T17:53:45.079ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
38312024-08-22T17:53:45.079ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } dns_group = internal
38322024-08-22T17:53:45.079ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 6.560854ms iteration = 5 status = {"addresses":["[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:3::1]:5353"]}
38332024-08-22T17:53:45.079ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
38342024-08-22T17:53:45.079ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
38352024-08-22T17:53:45.081ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38362024-08-22T17:53:45.081ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38372024-08-22T17:53:45.082ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-c961ede4-7532-4ca1-95a5-690b4d7c9b5a.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-c961ede4-7532-4ca1-95a5-690b4d7c9b5a.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1813, tv_nsec: 610762365 } })
38382024-08-22T17:53:45.082ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1050
38392024-08-22T17:53:45.083ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
38402024-08-22T17:53:45.083ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
38412024-08-22T17:53:45.083ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
38422024-08-22T17:53:45.084ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-c961ede4-7532-4ca1-95a5-690b4d7c9b5a.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-c961ede4-7532-4ca1-95a5-690b4d7c9b5a.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1813, tv_nsec: 612460763 } })
38432024-08-22T17:53:45.084ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1050
38442024-08-22T17:53:45.085ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
38452024-08-22T17:53:45.085ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
38462024-08-22T17:53:45.085ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
38472024-08-22T17:53:45.087ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
38482024-08-22T17:53:45.087ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
38492024-08-22T17:53:45.088ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38502024-08-22T17:53:45.088ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38512024-08-22T17:53:45.089ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38522024-08-22T17:53:45.091ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
38532024-08-22T17:53:45.091ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
38542024-08-22T17:53:45.091ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
38552024-08-22T17:53:45.092ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
38562024-08-22T17:53:45.092ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
38572024-08-22T17:53:45.093ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
38582024-08-22T17:53:45.093ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
38592024-08-22T17:53:45.093ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
38602024-08-22T17:53:45.093ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
38612024-08-22T17:53:45.093ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2024-08-22 17:43:45.072988520 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
38622024-08-22T17:53:45.093ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 20.890908ms iteration = 5 status = {"expiration":"2024-08-22T17:43:45.072988520Z","pruned":[]}
38632024-08-22T17:53:45.097ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38642024-08-22T17:53:45.099ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
38652024-08-22T17:53:45.099ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
38662024-08-22T17:53:45.101ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
38672024-08-22T17:53:45.101ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
38682024-08-22T17:53:45.101ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
38692024-08-22T17:53:45.103ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
38702024-08-22T17:53:45.103ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
38712024-08-22T17:53:45.103ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
38722024-08-22T17:53:45.103ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
38732024-08-22T17:53:45.103ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-c961ede4-7532-4ca1-95a5-690b4d7c9b5a.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-c961ede4-7532-4ca1-95a5-690b4d7c9b5a.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1813, tv_nsec: 632145945 } })
38742024-08-22T17:53:45.104ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38752024-08-22T17:53:45.104ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1050 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561
38762024-08-22T17:53:45.105ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38772024-08-22T17:53:45.107ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
38782024-08-22T17:53:45.107ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
38792024-08-22T17:53:45.108ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
38802024-08-22T17:53:45.108ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
38812024-08-22T17:53:45.108ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
38822024-08-22T17:53:45.108ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
38832024-08-22T17:53:45.114ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 41.394783ms iteration = 10 status = {"physical_disks_added":0}
38842024-08-22T17:53:45.130ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:55
38852024-08-22T17:53:45.130ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
38862024-08-22T17:53:45.130ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
38872024-08-22T17:53:45.130ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
38882024-08-22T17:53:45.141ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0
38892024-08-22T17:53:45.141ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:189 found = 0 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0
38902024-08-22T17:53:45.141ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 68.378773ms iteration = 5 status = {"error":null,"error_count":0,"found":0,"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0}
38912024-08-22T17:53:45.144ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
38922024-08-22T17:53:45.144ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
38932024-08-22T17:53:45.144ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
38942024-08-22T17:53:45.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
38952024-08-22T17:53:45.145ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38962024-08-22T17:53:45.146ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
38972024-08-22T17:53:45.147ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
38982024-08-22T17:53:45.148ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
38992024-08-22T17:53:45.148ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
39002024-08-22T17:53:45.148ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
39012024-08-22T17:53:45.148ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
39022024-08-22T17:53:45.148ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
39032024-08-22T17:53:45.150ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39042024-08-22T17:53:45.150ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39052024-08-22T17:53:45.151ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
39062024-08-22T17:53:45.152ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
39072024-08-22T17:53:45.152ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39082024-08-22T17:53:45.155ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 80.452776ms iteration = 9 status = {"errors":[],"garbage_collect_requested":[]}
39092024-08-22T17:53:45.155ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
39102024-08-22T17:53:45.155ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
39112024-08-22T17:53:45.155ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
39122024-08-22T17:53:45.156ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: c961ede4-7532-4ca1-95a5-690b4d7c9b5a, lookup_type: ById(c961ede4-7532-4ca1-95a5-690b4d7c9b5a) } result = Ok(())
39132024-08-22T17:53:45.156ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
39142024-08-22T17:53:45.156ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
39152024-08-22T17:53:45.156ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
39162024-08-22T17:53:45.158ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
39172024-08-22T17:53:45.159ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 85.965204ms iteration = 9 status = 0
39182024-08-22T17:53:45.159ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
39192024-08-22T17:53:45.159ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
39202024-08-22T17:53:45.159ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
39212024-08-22T17:53:45.159ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
39222024-08-22T17:53:45.159ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
39232024-08-22T17:53:45.159ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
39242024-08-22T17:53:45.161ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39252024-08-22T17:53:45.161ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39262024-08-22T17:53:45.165ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 147bdc10-344c-4ee8-a219-603cfc66e6d2, lookup_type: ById(147bdc10-344c-4ee8-a219-603cfc66e6d2) } result = Ok(())
39272024-08-22T17:53:45.165ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
39282024-08-22T17:53:45.165ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
39292024-08-22T17:53:45.165ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
39302024-08-22T17:53:45.165ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
39312024-08-22T17:53:45.165ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
39322024-08-22T17:53:45.186ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 147bdc10-344c-4ee8-a219-603cfc66e6d2 original_time_created = 2024-08-22 17:49:18.445237 UTC target_id = 147bdc10-344c-4ee8-a219-603cfc66e6d2 time_created = 2024-08-22 17:49:18.445237 UTC
39332024-08-22T17:53:45.186ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 113.863468ms iteration = 25 status = {"status":"target blueprint unchanged","target_id":"147bdc10-344c-4ee8-a219-603cfc66e6d2","time_created":"2024-08-22T17:49:18.445237Z"}
39342024-08-22T17:53:45.191ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:55
39352024-08-22T17:53:45.191ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:220 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
39362024-08-22T17:53:45.191ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 119.007375ms iteration = 9 status = {"destroyed_active_vmms":0,"error":null,"saga_start_failures":0,"sagas_completed":0,"sagas_failed":0,"sagas_started":0,"terminated_active_migrations":0}
39372024-08-22T17:53:45.193ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 120.540393ms iteration = 9 status = {"region_replacement_started_err":0,"region_replacement_started_ok":0}
39382024-08-22T17:53:45.193ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:454 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
39392024-08-22T17:53:45.193ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 121.232346ms iteration = 9 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
39402024-08-22T17:53:45.195ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
39412024-08-22T17:53:45.195ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:140 found = 0 not_ready_to_be_deleted = 0
39422024-08-22T17:53:45.195ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 122.134531ms iteration = 5 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
39432024-08-22T17:53:45.197ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
39442024-08-22T17:53:45.197ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
39452024-08-22T17:53:45.198ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
39462024-08-22T17:53:45.198ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
39472024-08-22T17:53:45.198ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
39482024-08-22T17:53:45.200ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 127.449698ms iteration = 9 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
39492024-08-22T17:53:45.200ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39502024-08-22T17:53:45.204ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 131.993812ms iteration = 25 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
39512024-08-22T17:53:45.204ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
39522024-08-22T17:53:45.204ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
39532024-08-22T17:53:45.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
39542024-08-22T17:53:45.219ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
39552024-08-22T17:53:45.219ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = d6f65b4e-2e87-480a-af21-a69b4520828c dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
39562024-08-22T17:53:45.219ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
39572024-08-22T17:53:45.219ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
39582024-08-22T17:53:45.223ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39592024-08-22T17:53:45.226ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
39602024-08-22T17:53:45.227ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
39612024-08-22T17:53:45.227ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
39622024-08-22T17:53:45.229ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = d6f65b4e-2e87-480a-af21-a69b4520828c dns_zone_name = control-plane.oxide.internal found_so_far = 50 version = 1
39632024-08-22T17:53:45.229ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
39642024-08-22T17:53:45.229ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-08-22 17:49:57.683168 UTC dns_group = internal generation = 1
39652024-08-22T17:53:45.229ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 156.68902ms iteration = 5 status = {"generation":1}
39662024-08-22T17:53:45.242ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
39672024-08-22T17:53:45.242ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 7f16ce95-fbde-4896-9a22-7f61d8d01c5d dns_zone_name = oxide.test found_so_far = 0 version = 2
39682024-08-22T17:53:45.242ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
39692024-08-22T17:53:45.242ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
39702024-08-22T17:53:45.243ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39712024-08-22T17:53:45.246ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
39722024-08-22T17:53:45.246ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
39732024-08-22T17:53:45.246ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
39742024-08-22T17:53:45.249ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 7f16ce95-fbde-4896-9a22-7f61d8d01c5d dns_zone_name = oxide.test found_so_far = 1 version = 2
39752024-08-22T17:53:45.249ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
39762024-08-22T17:53:45.249ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-08-22 17:49:58.502032 UTC dns_group = external generation = 2
39772024-08-22T17:53:45.249ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 175.973769ms iteration = 5 status = {"generation":2}
39782024-08-22T17:53:45.261ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
39792024-08-22T17:53:45.262ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
39802024-08-22T17:53:45.263ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39812024-08-22T17:53:45.263ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1058 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 zone_address = fd00:1122:3344:101::2
39822024-08-22T17:53:45.263ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
39832024-08-22T17:53:45.264ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "6a2d2726-c7bf-4b88-b962-dd16b1888462", "content-length": "26", "date": "Thu, 22 Aug 2024 17:53:44 GMT"} })
39842024-08-22T17:53:45.264ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1063 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
39852024-08-22T17:53:45.265ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1095 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561
39862024-08-22T17:53:45.265ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
39872024-08-22T17:53:45.265ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
39882024-08-22T17:53:45.266ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
39892024-08-22T17:53:45.284ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
39902024-08-22T17:53:45.284ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
39912024-08-22T17:53:45.286ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39922024-08-22T17:53:45.288ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
39932024-08-22T17:53:45.288ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
39942024-08-22T17:53:45.288ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
39952024-08-22T17:53:45.289ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39962024-08-22T17:53:45.291ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
39972024-08-22T17:53:45.291ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
39982024-08-22T17:53:45.291ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
39992024-08-22T17:53:45.300ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-08-22T17:49:43.060798Z, time_modified: 2024-08-22T17:49:43.060798Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 10d39b39-43d3-41db-b622-3ef3e09e2dad, db_silo: Silo { identity: SiloIdentity { id: 10d39b39-43d3-41db-b622-3ef3e09e2dad, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-08-22T17:49:58.442383Z, time_modified: 2024-08-22T17:49:58.442383Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "852177ddd7117113b778290a63910551c7b25a6718e9c8fda4590027c9443fee" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 10d39b39-43d3-41db-b622-3ef3e09e2dad, db_silo: Silo { identity: SiloIdentity { id: 10d39b39-43d3-41db-b622-3ef3e09e2dad, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-08-22T17:49:58.442383Z, time_modified: 2024-08-22T17:49:58.442383Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "852177ddd7117113b778290a63910551c7b25a6718e9c8fda4590027c9443fee" }] }) }
40002024-08-22T17:53:45.300ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 227.736187ms iteration = 5 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"10d39b39-43d3-41db-b622-3ef3e09e2dad","tls_certs":["852177ddd7117113b778290a63910551c7b25a6718e9c8fda4590027c9443fee"]}},"default_endpoint":{"silo_id":"10d39b39-43d3-41db-b622-3ef3e09e2dad","tls_certs":["852177ddd7117113b778290a63910551c7b25a6718e9c8fda4590027c9443fee"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
40012024-08-22T17:53:45.323ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
40022024-08-22T17:53:45.336ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
40032024-08-22T17:53:45.414ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1058 zone_address = fd00:1122:3344:101::2
40042024-08-22T17:53:45.414ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
40052024-08-22T17:53:45.414ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "dcb445a2-c9f5-4f3c-94bd-9a11122c1b27", "content-length": "26", "date": "Thu, 22 Aug 2024 17:53:44 GMT"} })
40062024-08-22T17:53:45.414ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1063 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
40072024-08-22T17:53:45.415ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1095 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
40082024-08-22T17:53:45.443ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1058 zone_address = fd00:1122:3344:101::2
40092024-08-22T17:53:45.443ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
40102024-08-22T17:53:45.443ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "b06c645e-7432-4cb9-b47c-3a39e6af5812", "content-length": "26", "date": "Thu, 22 Aug 2024 17:53:44 GMT"} })
40112024-08-22T17:53:45.443ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1063 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
40122024-08-22T17:53:45.445ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1095 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
40132024-08-22T17:53:45.445ZDEBGnexus (BackgroundTasks): client request SledAgent = c961ede4-7532-4ca1-95a5-690b4d7c9b5a background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
40142024-08-22T17:53:45.446ZDEBGnexus (BackgroundTasks): client response SledAgent = c961ede4-7532-4ca1-95a5-690b4d7c9b5a background_task = vpc_route_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/vpc-routes", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "4be6df07-2e75-4aab-b7f1-83b1fe37acf2", "content-length": "372", "date": "Thu, 22 Aug 2024 17:53:44 GMT"} })
40152024-08-22T17:53:45.446ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
40162024-08-22T17:53:45.446ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
40172024-08-22T17:53:45.448ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
40182024-08-22T17:53:45.448ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
40192024-08-22T17:53:45.451ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
40202024-08-22T17:53:45.452ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
40212024-08-22T17:53:45.455ZDEBGnexus (BackgroundTasks): client request SledAgent = c961ede4-7532-4ca1-95a5-690b4d7c9b5a background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
40222024-08-22T17:53:45.455ZDEBGnexus (BackgroundTasks): client response SledAgent = c961ede4-7532-4ca1-95a5-690b4d7c9b5a background_task = vpc_route_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/vpc-routes", query: None, fragment: None }, status: 204, headers: {"x-request-id": "fa83eb19-6090-4624-851b-416c88b33704", "date": "Thu, 22 Aug 2024 17:53:44 GMT"} })
40232024-08-22T17:53:45.457ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 384.493197ms iteration = 9 status = {}
40242024-08-22T17:53:45.479ZDEBGnexus (BackgroundTasks): client request SledAgent = c961ede4-7532-4ca1-95a5-690b4d7c9b5a background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
40252024-08-22T17:53:45.480ZDEBGnexus (BackgroundTasks): client response SledAgent = c961ede4-7532-4ca1-95a5-690b4d7c9b5a background_task = v2p_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/v2p", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "70bfc3e3-5b03-409b-8ab0-21673edca97b", "content-length": "691", "date": "Thu, 22 Aug 2024 17:53:44 GMT"} })
40262024-08-22T17:53:45.480ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = buskin
40272024-08-22T17:53:45.480ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = buskin
40282024-08-22T17:53:45.480ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = buskin
40292024-08-22T17:53:45.482ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 409.413875ms iteration = 9 status = {}
40302024-08-22T17:53:45.547ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
40312024-08-22T17:53:45.549ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::8)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "fc4370c8-60e7-4a47-9a96-4a97d7a0f219", "date": "Thu, 22 Aug 2024 17:53:45 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
40322024-08-22T17:53:45.549ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
40332024-08-22T17:53:45.551ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::9)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "450b6351-d491-4a4d-945c-59e89ebb107c", "date": "Thu, 22 Aug 2024 17:53:45 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
40342024-08-22T17:53:45.551ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
40352024-08-22T17:53:45.552ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
40362024-08-22T17:53:45.553ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 480.291902ms iteration = 7 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
40372024-08-22T17:53:45.555ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
40382024-08-22T17:53:45.555ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "5031fd1d-09c7-4ac3-b1a9-98f16005cdbf", "content-length": "1", "date": "Thu, 22 Aug 2024 17:53:45 GMT"} })
40392024-08-22T17:53:45.560ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
40402024-08-22T17:53:45.560ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
40412024-08-22T17:53:45.562ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 489.953842ms iteration = 9 status = null
40422024-08-22T17:53:45.593ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 520.288759ms iteration = 9 status = {}
40432024-08-22T17:53:45.620ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
40442024-08-22T17:53:45.755ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
40452024-08-22T17:53:45.755ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:3::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "e0465cf2-ff86-4048-afa8-5aa24a6fc153", "date": "Thu, 22 Aug 2024 17:53:45 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
40462024-08-22T17:53:45.756ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
40472024-08-22T17:53:45.757ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:1::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "4909e9f6-5f3a-4a98-878f-7246d8268e88", "date": "Thu, 22 Aug 2024 17:53:45 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
40482024-08-22T17:53:45.757ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
40492024-08-22T17:53:45.758ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:2::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "3e3e6878-34df-4f9b-b72d-b8d4034b7f0e", "date": "Thu, 22 Aug 2024 17:53:45 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
40502024-08-22T17:53:45.758ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
40512024-08-22T17:53:45.760ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
40522024-08-22T17:53:45.760ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 687.694154ms iteration = 7 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
40532024-08-22T17:53:45.777ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
40542024-08-22T17:53:45.777ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
40552024-08-22T17:53:45.779ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:147 port = SwitchPort { id: c772d372-7448-4ce0-b8d1-efa645d18ed1, rack_id: c66d93a6-e2c3-4b19-9564-3b58bad50561, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(9ff05247-84aa-47d8-8619-ce0a8dbbaf7c) } rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 switch_location = Switch0
40562024-08-22T17:53:45.779ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
40572024-08-22T17:53:45.779ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
40582024-08-22T17:53:45.790ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
40592024-08-22T17:53:45.790ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/port/qsfp0/settings", query: Some("tag=omicron"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "61efcbc9-325c-47e8-a9e1-e7c36ad3e458", "content-length": "121", "date": "Thu, 22 Aug 2024 17:53:45 GMT"} })
40602024-08-22T17:53:45.790ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1714 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G } }} } switch_port_id = "qsfp0"
40612024-08-22T17:53:45.790ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1741 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 9ff05247-84aa-47d8-8619-ce0a8dbbaf7c
40622024-08-22T17:53:45.790ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:379 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
40632024-08-22T17:53:45.790ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
40642024-08-22T17:53:45.790ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:392 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 routes = {}
40652024-08-22T17:53:45.790ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:398 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 routes = {}
40662024-08-22T17:53:45.790ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:418 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561
40672024-08-22T17:53:45.790ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
40682024-08-22T17:53:45.791ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
40692024-08-22T17:53:45.793ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
40702024-08-22T17:53:45.794ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv4", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "13cc5497-609f-436c-b967-d54ec40a9a74", "content-length": "2", "date": "Thu, 22 Aug 2024 17:53:45 GMT"} })
40712024-08-22T17:53:45.794ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
40722024-08-22T17:53:45.794ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "06fe0807-4038-413b-8d11-2b7f7e0f51d7", "content-length": "52", "date": "Thu, 22 Aug 2024 17:53:45 GMT"} })
40732024-08-22T17:53:45.794ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0" }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:467 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 switch_location = Switch0
40742024-08-22T17:53:45.794ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
40752024-08-22T17:53:45.894ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/switch-ports", query: None, fragment: None }, status: 204, headers: {"x-request-id": "85c704ef-28f0-4470-a1c4-5162d3dccd5a", "date": "Thu, 22 Aug 2024 17:53:45 GMT"} })
40762024-08-22T17:53:45.894ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:770 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 switch = Switch0
40772024-08-22T17:53:45.894ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
40782024-08-22T17:53:45.894ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/network-bootstore-config", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "2151a16c-0457-475b-99cd-8741b7f56f02", "content-length": "667", "date": "Thu, 22 Aug 2024 17:53:45 GMT"} })
40792024-08-22T17:53:45.894ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
40802024-08-22T17:53:45.894ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
40812024-08-22T17:53:45.897ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
40822024-08-22T17:53:45.898ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
40832024-08-22T17:53:45.901ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
40842024-08-22T17:53:45.902ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
40852024-08-22T17:53:45.903ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
40862024-08-22T17:53:45.904ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
40872024-08-22T17:53:45.910ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 837.610959ms iteration = 9 status = {}
40882024-08-22T17:53:46.006ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 245 local_addr = [fd00:1122:3344:101::c]:56994 method = GET remote_addr = [fd00:1122:3344:101::d]:40670 req_id = 0f01a672-5283-40d6-8506-2619246da1f8 response_code = 200 uri = /eaa260db-2b19-4ae1-97e8-5735c9f5d955
40892024-08-22T17:53:55.073ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 26 reason = Timeout
40902024-08-22T17:53:55.073ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 26 reason = Timeout
40912024-08-22T17:53:55.073ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
40922024-08-22T17:53:55.073ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
40932024-08-22T17:53:55.073ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
40942024-08-22T17:53:55.073ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
40952024-08-22T17:53:55.075ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
40962024-08-22T17:53:55.075ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
40972024-08-22T17:53:55.075ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40982024-08-22T17:53:55.077ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.784179ms iteration = 26 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
40992024-08-22T17:53:55.078ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
41002024-08-22T17:53:55.078ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
41012024-08-22T17:53:55.078ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
41022024-08-22T17:53:55.080ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
41032024-08-22T17:53:55.080ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
41042024-08-22T17:53:55.082ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41052024-08-22T17:53:55.084ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 147bdc10-344c-4ee8-a219-603cfc66e6d2, lookup_type: ById(147bdc10-344c-4ee8-a219-603cfc66e6d2) } result = Ok(())
41062024-08-22T17:53:55.084ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
41072024-08-22T17:53:55.084ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
41082024-08-22T17:53:55.092ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 147bdc10-344c-4ee8-a219-603cfc66e6d2 original_time_created = 2024-08-22 17:49:18.445237 UTC target_id = 147bdc10-344c-4ee8-a219-603cfc66e6d2 time_created = 2024-08-22 17:49:18.445237 UTC
41092024-08-22T17:53:55.092ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 19.073819ms iteration = 26 status = {"status":"target blueprint unchanged","target_id":"147bdc10-344c-4ee8-a219-603cfc66e6d2","time_created":"2024-08-22T17:49:18.445237Z"}
41102024-08-22T17:53:56.007ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 213 local_addr = [fd00:1122:3344:101::c]:56994 method = GET remote_addr = [fd00:1122:3344:101::d]:40670 req_id = b1a665df-4d7b-4b02-a938-9d774b5b3f72 response_code = 200 uri = /eaa260db-2b19-4ae1-97e8-5735c9f5d955
41112024-08-22T17:54:05.073ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 27 reason = Timeout
41122024-08-22T17:54:05.073ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
41132024-08-22T17:54:05.073ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 27 reason = Timeout
41142024-08-22T17:54:05.073ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
41152024-08-22T17:54:05.073ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
41162024-08-22T17:54:05.073ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
41172024-08-22T17:54:05.075ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41182024-08-22T17:54:05.077ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
41192024-08-22T17:54:05.077ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
41202024-08-22T17:54:05.078ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
41212024-08-22T17:54:05.079ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
41222024-08-22T17:54:05.080ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
41232024-08-22T17:54:05.081ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41242024-08-22T17:54:05.083ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 147bdc10-344c-4ee8-a219-603cfc66e6d2, lookup_type: ById(147bdc10-344c-4ee8-a219-603cfc66e6d2) } result = Ok(())
41252024-08-22T17:54:05.083ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
41262024-08-22T17:54:05.083ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
41272024-08-22T17:54:05.094ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 147bdc10-344c-4ee8-a219-603cfc66e6d2 original_time_created = 2024-08-22 17:49:18.445237 UTC target_id = 147bdc10-344c-4ee8-a219-603cfc66e6d2 time_created = 2024-08-22 17:49:18.445237 UTC
41282024-08-22T17:54:05.094ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 20.801467ms iteration = 27 status = {"status":"target blueprint unchanged","target_id":"147bdc10-344c-4ee8-a219-603cfc66e6d2","time_created":"2024-08-22T17:49:18.445237Z"}
41292024-08-22T17:54:05.108ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
41302024-08-22T17:54:05.108ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
41312024-08-22T17:54:05.109ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 36.289068ms iteration = 27 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
41322024-08-22T17:54:06.006ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 225 local_addr = [fd00:1122:3344:101::c]:56994 method = GET remote_addr = [fd00:1122:3344:101::d]:40670 req_id = 37ce59bf-c934-441a-b961-f13b05525f19 response_code = 200 uri = /eaa260db-2b19-4ae1-97e8-5735c9f5d955
41332024-08-22T17:54:15.073ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 10 reason = Timeout
41342024-08-22T17:54:15.073ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 10 reason = Timeout
41352024-08-22T17:54:15.073ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
41362024-08-22T17:54:15.073ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
41372024-08-22T17:54:15.073ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 10 reason = Timeout
41382024-08-22T17:54:15.073ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 10 reason = Timeout
41392024-08-22T17:54:15.073ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
41402024-08-22T17:54:15.073ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
41412024-08-22T17:54:15.073ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 10 reason = Timeout
41422024-08-22T17:54:15.073ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 28 reason = Timeout
41432024-08-22T17:54:15.073ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
41442024-08-22T17:54:15.073ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 10 reason = Timeout
41452024-08-22T17:54:15.073ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 28 reason = Timeout
41462024-08-22T17:54:15.073ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 10 reason = Timeout
41472024-08-22T17:54:15.073ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
41482024-08-22T17:54:15.073ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
41492024-08-22T17:54:15.073ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 10 reason = Timeout
41502024-08-22T17:54:15.073ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
41512024-08-22T17:54:15.073ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
41522024-08-22T17:54:15.073ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 10 reason = Timeout
41532024-08-22T17:54:15.073ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
41542024-08-22T17:54:15.073ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 10 reason = Timeout
41552024-08-22T17:54:15.073ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
41562024-08-22T17:54:15.073ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 10 reason = Timeout
41572024-08-22T17:54:15.073ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
41582024-08-22T17:54:15.073ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
41592024-08-22T17:54:15.073ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 11 reason = Timeout
41602024-08-22T17:54:15.073ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
41612024-08-22T17:54:15.073ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
41622024-08-22T17:54:15.073ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
41632024-08-22T17:54:15.073ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
41642024-08-22T17:54:15.073ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
41652024-08-22T17:54:15.073ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
41662024-08-22T17:54:15.073ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
41672024-08-22T17:54:15.073ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
41682024-08-22T17:54:15.073ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
41692024-08-22T17:54:15.073ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
41702024-08-22T17:54:15.073ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
41712024-08-22T17:54:15.073ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
41722024-08-22T17:54:15.073ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
41732024-08-22T17:54:15.073ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
41742024-08-22T17:54:15.075ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 10 reason = Timeout
41752024-08-22T17:54:15.075ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
41762024-08-22T17:54:15.075ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
41772024-08-22T17:54:15.076ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-c961ede4-7532-4ca1-95a5-690b4d7c9b5a.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-c961ede4-7532-4ca1-95a5-690b4d7c9b5a.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1843, tv_nsec: 604117312 } })
41782024-08-22T17:54:15.076ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41792024-08-22T17:54:15.077ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41802024-08-22T17:54:15.077ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1050
41812024-08-22T17:54:15.077ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
41822024-08-22T17:54:15.077ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-c961ede4-7532-4ca1-95a5-690b4d7c9b5a.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-c961ede4-7532-4ca1-95a5-690b4d7c9b5a.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1843, tv_nsec: 605524299 } })
41832024-08-22T17:54:15.078ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
41842024-08-22T17:54:15.078ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1050
41852024-08-22T17:54:15.079ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41862024-08-22T17:54:15.080ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
41872024-08-22T17:54:15.080ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
41882024-08-22T17:54:15.080ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
41892024-08-22T17:54:15.080ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41902024-08-22T17:54:15.080ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
41912024-08-22T17:54:15.080ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
41922024-08-22T17:54:15.080ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
41932024-08-22T17:54:15.081ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 7.921901ms iteration = 10 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
41942024-08-22T17:54:15.081ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
41952024-08-22T17:54:15.081ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
41962024-08-22T17:54:15.081ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
41972024-08-22T17:54:15.082ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41982024-08-22T17:54:15.083ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
41992024-08-22T17:54:15.083ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
42002024-08-22T17:54:15.083ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
42012024-08-22T17:54:15.084ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
42022024-08-22T17:54:15.084ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
42032024-08-22T17:54:15.084ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
42042024-08-22T17:54:15.085ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42052024-08-22T17:54:15.086ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
42062024-08-22T17:54:15.087ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
42072024-08-22T17:54:15.087ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
42082024-08-22T17:54:15.087ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
42092024-08-22T17:54:15.087ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
42102024-08-22T17:54:15.087ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
42112024-08-22T17:54:15.087ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
42122024-08-22T17:54:15.089ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
42132024-08-22T17:54:15.090ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
42142024-08-22T17:54:15.090ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42152024-08-22T17:54:15.091ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-c961ede4-7532-4ca1-95a5-690b4d7c9b5a.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-c961ede4-7532-4ca1-95a5-690b4d7c9b5a.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1843, tv_nsec: 619065709 } })
42162024-08-22T17:54:15.092ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1050 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561
42172024-08-22T17:54:15.093ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
42182024-08-22T17:54:15.093ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
42192024-08-22T17:54:15.093ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
42202024-08-22T17:54:15.093ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
42212024-08-22T17:54:15.093ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
42222024-08-22T17:54:15.097ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42232024-08-22T17:54:15.097ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:454 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
42242024-08-22T17:54:15.097ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 24.030454ms iteration = 10 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
42252024-08-22T17:54:15.099ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42262024-08-22T17:54:15.099ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 147bdc10-344c-4ee8-a219-603cfc66e6d2, lookup_type: ById(147bdc10-344c-4ee8-a219-603cfc66e6d2) } result = Ok(())
42272024-08-22T17:54:15.099ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
42282024-08-22T17:54:15.099ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
42292024-08-22T17:54:15.102ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
42302024-08-22T17:54:15.102ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
42312024-08-22T17:54:15.102ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
42322024-08-22T17:54:15.108ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 35.287442ms iteration = 11 status = {"physical_disks_added":0}
42332024-08-22T17:54:15.110ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
42342024-08-22T17:54:15.110ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 147bdc10-344c-4ee8-a219-603cfc66e6d2 original_time_created = 2024-08-22 17:49:18.445237 UTC target_id = 147bdc10-344c-4ee8-a219-603cfc66e6d2 time_created = 2024-08-22 17:49:18.445237 UTC
42352024-08-22T17:54:15.110ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 37.328493ms iteration = 28 status = {"status":"target blueprint unchanged","target_id":"147bdc10-344c-4ee8-a219-603cfc66e6d2","time_created":"2024-08-22T17:49:18.445237Z"}
42362024-08-22T17:54:15.110ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
42372024-08-22T17:54:15.113ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
42382024-08-22T17:54:15.113ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
42392024-08-22T17:54:15.115ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42402024-08-22T17:54:15.118ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: c961ede4-7532-4ca1-95a5-690b4d7c9b5a, lookup_type: ById(c961ede4-7532-4ca1-95a5-690b4d7c9b5a) } result = Ok(())
42412024-08-22T17:54:15.118ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
42422024-08-22T17:54:15.118ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
42432024-08-22T17:54:15.120ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
42442024-08-22T17:54:15.120ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 47.363845ms iteration = 10 status = 0
42452024-08-22T17:54:15.121ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
42462024-08-22T17:54:15.126ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 53.300406ms iteration = 28 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
42472024-08-22T17:54:15.133ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:55
42482024-08-22T17:54:15.133ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 59.662719ms iteration = 10 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
42492024-08-22T17:54:15.133ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
42502024-08-22T17:54:15.133ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
42512024-08-22T17:54:15.133ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
42522024-08-22T17:54:15.133ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
42532024-08-22T17:54:15.133ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
42542024-08-22T17:54:15.134ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 58.618423ms iteration = 10 status = {"errors":[],"garbage_collect_requested":[]}
42552024-08-22T17:54:15.135ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:55
42562024-08-22T17:54:15.135ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:220 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
42572024-08-22T17:54:15.135ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 61.622708ms iteration = 10 status = {"destroyed_active_vmms":0,"error":null,"saga_start_failures":0,"sagas_completed":0,"sagas_failed":0,"sagas_started":0,"terminated_active_migrations":0}
42582024-08-22T17:54:15.151ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1058 zone_address = fd00:1122:3344:101::2
42592024-08-22T17:54:15.151ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
42602024-08-22T17:54:15.151ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "bbb11a52-7a3a-4ab4-a481-8f8ce0fa839d", "content-length": "26", "date": "Thu, 22 Aug 2024 17:54:14 GMT"} })
42612024-08-22T17:54:15.151ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1063 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
42622024-08-22T17:54:15.153ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1095 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
42632024-08-22T17:54:15.162ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 88.794009ms iteration = 10 status = {"region_replacement_started_err":0,"region_replacement_started_ok":0}
42642024-08-22T17:54:15.174ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1058 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 zone_address = fd00:1122:3344:101::2
42652024-08-22T17:54:15.174ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
42662024-08-22T17:54:15.174ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "5cbdfb88-2ee9-477b-980c-d2721069899e", "content-length": "26", "date": "Thu, 22 Aug 2024 17:54:14 GMT"} })
42672024-08-22T17:54:15.174ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1063 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
42682024-08-22T17:54:15.177ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1095 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561
42692024-08-22T17:54:15.186ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1058 zone_address = fd00:1122:3344:101::2
42702024-08-22T17:54:15.186ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
42712024-08-22T17:54:15.186ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "edc0f1e6-0762-45b1-a073-ca279ba2def1", "content-length": "26", "date": "Thu, 22 Aug 2024 17:54:14 GMT"} })
42722024-08-22T17:54:15.186ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1063 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
42732024-08-22T17:54:15.187ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1095 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
42742024-08-22T17:54:15.235ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
42752024-08-22T17:54:15.235ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
42762024-08-22T17:54:15.237ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42772024-08-22T17:54:15.239ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
42782024-08-22T17:54:15.239ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
42792024-08-22T17:54:15.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
42802024-08-22T17:54:15.241ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42812024-08-22T17:54:15.242ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
42822024-08-22T17:54:15.242ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
42832024-08-22T17:54:15.242ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
42842024-08-22T17:54:15.270ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 197.097398ms iteration = 10 status = {}
42852024-08-22T17:54:15.274ZDEBGnexus (BackgroundTasks): client request SledAgent = c961ede4-7532-4ca1-95a5-690b4d7c9b5a background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
42862024-08-22T17:54:15.274ZDEBGnexus (BackgroundTasks): client response SledAgent = c961ede4-7532-4ca1-95a5-690b4d7c9b5a background_task = vpc_route_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/vpc-routes", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "e95a769e-b638-4769-837c-96293504b909", "content-length": "372", "date": "Thu, 22 Aug 2024 17:54:14 GMT"} })
42872024-08-22T17:54:15.274ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
42882024-08-22T17:54:15.275ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
42892024-08-22T17:54:15.276ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
42902024-08-22T17:54:15.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
42912024-08-22T17:54:15.279ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
42922024-08-22T17:54:15.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
42932024-08-22T17:54:15.282ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
42942024-08-22T17:54:15.282ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "d20c4b82-fa40-4b11-b59c-4c0d0421f76c", "content-length": "1", "date": "Thu, 22 Aug 2024 17:54:14 GMT"} })
42952024-08-22T17:54:15.283ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
42962024-08-22T17:54:15.283ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
42972024-08-22T17:54:15.284ZDEBGnexus (BackgroundTasks): client request SledAgent = c961ede4-7532-4ca1-95a5-690b4d7c9b5a background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
42982024-08-22T17:54:15.284ZDEBGnexus (BackgroundTasks): client response SledAgent = c961ede4-7532-4ca1-95a5-690b4d7c9b5a background_task = vpc_route_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/vpc-routes", query: None, fragment: None }, status: 204, headers: {"x-request-id": "f507bba3-8da3-4c10-a367-ebbe7af2502a", "date": "Thu, 22 Aug 2024 17:54:14 GMT"} })
42992024-08-22T17:54:15.285ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 211.901764ms iteration = 10 status = {}
43002024-08-22T17:54:15.285ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 212.312317ms iteration = 10 status = null
43012024-08-22T17:54:15.351ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
43022024-08-22T17:54:15.352ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
43032024-08-22T17:54:15.367ZDEBGnexus (BackgroundTasks): client request SledAgent = c961ede4-7532-4ca1-95a5-690b4d7c9b5a background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
43042024-08-22T17:54:15.368ZDEBGnexus (BackgroundTasks): client response SledAgent = c961ede4-7532-4ca1-95a5-690b4d7c9b5a background_task = v2p_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/v2p", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "1c1a2a7a-9b75-4efa-a234-91eb200cc074", "content-length": "691", "date": "Thu, 22 Aug 2024 17:54:14 GMT"} })
43052024-08-22T17:54:15.368ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = buskin
43062024-08-22T17:54:15.368ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = buskin
43072024-08-22T17:54:15.368ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = buskin
43082024-08-22T17:54:15.368ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 295.366056ms iteration = 10 status = {}
43092024-08-22T17:54:15.369ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:147 port = SwitchPort { id: c772d372-7448-4ce0-b8d1-efa645d18ed1, rack_id: c66d93a6-e2c3-4b19-9564-3b58bad50561, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(9ff05247-84aa-47d8-8619-ce0a8dbbaf7c) } rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 switch_location = Switch0
43102024-08-22T17:54:15.369ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
43112024-08-22T17:54:15.370ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
43122024-08-22T17:54:15.378ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
43132024-08-22T17:54:15.378ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/port/qsfp0/settings", query: Some("tag=omicron"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "58cece81-ec56-4d94-bd1a-4e7194f31521", "content-length": "121", "date": "Thu, 22 Aug 2024 17:54:14 GMT"} })
43142024-08-22T17:54:15.378ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1714 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G } }} } switch_port_id = "qsfp0"
43152024-08-22T17:54:15.378ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1741 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 9ff05247-84aa-47d8-8619-ce0a8dbbaf7c
43162024-08-22T17:54:15.378ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:379 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
43172024-08-22T17:54:15.378ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
43182024-08-22T17:54:15.378ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:392 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 routes = {}
43192024-08-22T17:54:15.378ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:398 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 routes = {}
43202024-08-22T17:54:15.378ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:418 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561
43212024-08-22T17:54:15.378ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
43222024-08-22T17:54:15.379ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
43232024-08-22T17:54:15.381ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
43242024-08-22T17:54:15.381ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv4", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "d1688c02-c933-4125-a852-fcc1f3d11bf7", "content-length": "2", "date": "Thu, 22 Aug 2024 17:54:14 GMT"} })
43252024-08-22T17:54:15.381ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
43262024-08-22T17:54:15.381ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "ed2c68c5-2ccf-4b64-9726-92d7b6b746f7", "content-length": "52", "date": "Thu, 22 Aug 2024 17:54:14 GMT"} })
43272024-08-22T17:54:15.381ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0" }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:467 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 switch_location = Switch0
43282024-08-22T17:54:15.381ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
43292024-08-22T17:54:15.470ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/switch-ports", query: None, fragment: None }, status: 204, headers: {"x-request-id": "a798e06a-8117-4c77-8b65-3f370573b9b1", "date": "Thu, 22 Aug 2024 17:54:14 GMT"} })
43302024-08-22T17:54:15.470ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:770 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 switch = Switch0
43312024-08-22T17:54:15.471ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
43322024-08-22T17:54:15.584ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/network-bootstore-config", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "cd71d449-3140-418a-88bc-e111e0381526", "content-length": "667", "date": "Thu, 22 Aug 2024 17:54:15 GMT"} })
43332024-08-22T17:54:15.584ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
43342024-08-22T17:54:15.584ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
43352024-08-22T17:54:15.587ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
43362024-08-22T17:54:15.588ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
43372024-08-22T17:54:15.591ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
43382024-08-22T17:54:15.591ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
43392024-08-22T17:54:15.593ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
43402024-08-22T17:54:15.593ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
43412024-08-22T17:54:15.600ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 527.496246ms iteration = 10 status = {}
43422024-08-22T17:54:16.007ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 206 local_addr = [fd00:1122:3344:101::c]:56994 method = GET remote_addr = [fd00:1122:3344:101::d]:40670 req_id = c4d23017-2002-493c-8217-cf5667650bc4 response_code = 200 uri = /eaa260db-2b19-4ae1-97e8-5735c9f5d955
43432024-08-22T17:54:25.073ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 29 reason = Timeout
43442024-08-22T17:54:25.073ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
43452024-08-22T17:54:25.073ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 29 reason = Timeout
43462024-08-22T17:54:25.073ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
43472024-08-22T17:54:25.073ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
43482024-08-22T17:54:25.073ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
43492024-08-22T17:54:25.075ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43502024-08-22T17:54:25.077ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
43512024-08-22T17:54:25.077ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
43522024-08-22T17:54:25.078ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
43532024-08-22T17:54:25.080ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
43542024-08-22T17:54:25.080ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
43552024-08-22T17:54:25.082ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43562024-08-22T17:54:25.084ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 147bdc10-344c-4ee8-a219-603cfc66e6d2, lookup_type: ById(147bdc10-344c-4ee8-a219-603cfc66e6d2) } result = Ok(())
43572024-08-22T17:54:25.084ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
43582024-08-22T17:54:25.084ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
43592024-08-22T17:54:25.093ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 147bdc10-344c-4ee8-a219-603cfc66e6d2 original_time_created = 2024-08-22 17:49:18.445237 UTC target_id = 147bdc10-344c-4ee8-a219-603cfc66e6d2 time_created = 2024-08-22 17:49:18.445237 UTC
43602024-08-22T17:54:25.093ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 20.090564ms iteration = 29 status = {"status":"target blueprint unchanged","target_id":"147bdc10-344c-4ee8-a219-603cfc66e6d2","time_created":"2024-08-22T17:49:18.445237Z"}
43612024-08-22T17:54:25.101ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
43622024-08-22T17:54:25.101ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
43632024-08-22T17:54:25.102ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 29.239211ms iteration = 29 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
43642024-08-22T17:54:26.007ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 210 local_addr = [fd00:1122:3344:101::c]:56994 method = GET remote_addr = [fd00:1122:3344:101::d]:40670 req_id = 6c397f95-5fea-4454-a742-b61f664b3236 response_code = 200 uri = /eaa260db-2b19-4ae1-97e8-5735c9f5d955
43652024-08-22T17:54:35.074ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 30 reason = Timeout
43662024-08-22T17:54:35.074ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 30 reason = Timeout
43672024-08-22T17:54:35.074ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
43682024-08-22T17:54:35.074ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
43692024-08-22T17:54:35.074ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
43702024-08-22T17:54:35.074ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
43712024-08-22T17:54:35.076ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
43722024-08-22T17:54:35.077ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
43732024-08-22T17:54:35.077ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43742024-08-22T17:54:35.078ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.85238ms iteration = 30 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
43752024-08-22T17:54:35.079ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
43762024-08-22T17:54:35.079ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
43772024-08-22T17:54:35.079ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
43782024-08-22T17:54:35.081ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
43792024-08-22T17:54:35.081ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
43802024-08-22T17:54:35.082ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43812024-08-22T17:54:35.085ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 147bdc10-344c-4ee8-a219-603cfc66e6d2, lookup_type: ById(147bdc10-344c-4ee8-a219-603cfc66e6d2) } result = Ok(())
43822024-08-22T17:54:35.085ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
43832024-08-22T17:54:35.085ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
43842024-08-22T17:54:35.093ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 147bdc10-344c-4ee8-a219-603cfc66e6d2 original_time_created = 2024-08-22 17:49:18.445237 UTC target_id = 147bdc10-344c-4ee8-a219-603cfc66e6d2 time_created = 2024-08-22 17:49:18.445237 UTC
43852024-08-22T17:54:35.093ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 18.760117ms iteration = 30 status = {"status":"target blueprint unchanged","target_id":"147bdc10-344c-4ee8-a219-603cfc66e6d2","time_created":"2024-08-22T17:49:18.445237Z"}
43862024-08-22T17:54:36.007ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 212 local_addr = [fd00:1122:3344:101::c]:56994 method = GET remote_addr = [fd00:1122:3344:101::d]:40670 req_id = eca48cbf-7bf9-45d6-bf95-1a5ace713e29 response_code = 200 uri = /eaa260db-2b19-4ae1-97e8-5735c9f5d955
43872024-08-22T17:54:45.073ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 6 reason = Timeout
43882024-08-22T17:54:45.073ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 31 reason = Timeout
43892024-08-22T17:54:45.073ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 11 reason = Timeout
43902024-08-22T17:54:45.073ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
43912024-08-22T17:54:45.073ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 11 reason = Timeout
43922024-08-22T17:54:45.073ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
43932024-08-22T17:54:45.073ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
43942024-08-22T17:54:45.074ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
43952024-08-22T17:54:45.074ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 11 reason = Timeout
43962024-08-22T17:54:45.074ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
43972024-08-22T17:54:45.074ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 6 reason = Timeout
43982024-08-22T17:54:45.074ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2024-08-22 17:44:45.073922790 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
43992024-08-22T17:54:45.074ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
44002024-08-22T17:54:45.074ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 8 reason = Timeout
44012024-08-22T17:54:45.074ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 8 reason = Timeout
44022024-08-22T17:54:45.074ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 7 reason = Timeout
44032024-08-22T17:54:45.074ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:80 reason = blueprint disabled target_id = 147bdc10-344c-4ee8-a219-603cfc66e6d2
44042024-08-22T17:54:45.074ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 6 reason = Timeout
44052024-08-22T17:54:45.074ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
44062024-08-22T17:54:45.074ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
44072024-08-22T17:54:45.074ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 20.19\u{b5}s iteration = 7 status = {"error":"blueprint disabled","target_id":"147bdc10-344c-4ee8-a219-603cfc66e6d2"}
44082024-08-22T17:54:45.074ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 11 reason = Timeout
44092024-08-22T17:54:45.074ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 6 reason = Timeout
44102024-08-22T17:54:45.074ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 11 reason = Timeout
44112024-08-22T17:54:45.074ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 31 reason = Timeout
44122024-08-22T17:54:45.074ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 11 reason = Timeout
44132024-08-22T17:54:45.074ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
44142024-08-22T17:54:45.074ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
44152024-08-22T17:54:45.074ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
44162024-08-22T17:54:45.074ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 6 reason = Timeout
44172024-08-22T17:54:45.074ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 6 reason = Timeout
44182024-08-22T17:54:45.074ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
44192024-08-22T17:54:45.074ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
44202024-08-22T17:54:45.074ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 12 reason = Timeout
44212024-08-22T17:54:45.074ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 11 reason = Timeout
44222024-08-22T17:54:45.074ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 6 reason = Timeout
44232024-08-22T17:54:45.074ZINFOnexus (BackgroundTasks): Abandoned VMM reaper running background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:76
44242024-08-22T17:54:45.074ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
44252024-08-22T17:54:45.074ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
44262024-08-22T17:54:45.074ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
44272024-08-22T17:54:45.074ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 11 reason = Timeout
44282024-08-22T17:54:45.074ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
44292024-08-22T17:54:45.074ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 6 reason = Timeout
44302024-08-22T17:54:45.074ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 11 reason = Timeout
44312024-08-22T17:54:45.074ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
44322024-08-22T17:54:45.074ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
44332024-08-22T17:54:45.074ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
44342024-08-22T17:54:45.074ZDEBGnexus (BackgroundTasks): activating background_task = service_firewall_rule_propagation iteration = 2 reason = Timeout
44352024-08-22T17:54:45.074ZDEBGnexus (service-firewall-rule-progator): starting background task for service firewall rule propagation background_task = service_firewall_rule_propagation
44362024-08-22T17:54:45.074ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
44372024-08-22T17:54:45.074ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 6 reason = Timeout
44382024-08-22T17:54:45.074ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 11 reason = Timeout
44392024-08-22T17:54:45.074ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:56
44402024-08-22T17:54:45.074ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
44412024-08-22T17:54:45.074ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
44422024-08-22T17:54:45.074ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 11 reason = Timeout
44432024-08-22T17:54:45.074ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
44442024-08-22T17:54:45.074ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
44452024-08-22T17:54:45.074ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
44462024-08-22T17:54:45.074ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
44472024-08-22T17:54:45.074ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
44482024-08-22T17:54:45.074ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
44492024-08-22T17:54:45.074ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
44502024-08-22T17:54:45.074ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
44512024-08-22T17:54:45.074ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
44522024-08-22T17:54:45.074ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
44532024-08-22T17:54:45.074ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
44542024-08-22T17:54:45.074ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
44552024-08-22T17:54:45.074ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
44562024-08-22T17:54:45.074ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
44572024-08-22T17:54:45.074ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
44582024-08-22T17:54:45.074ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
44592024-08-22T17:54:45.074ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
44602024-08-22T17:54:45.074ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
44612024-08-22T17:54:45.074ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
44622024-08-22T17:54:45.074ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
44632024-08-22T17:54:45.074ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
44642024-08-22T17:54:45.074ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
44652024-08-22T17:54:45.075ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 11 reason = Timeout
44662024-08-22T17:54:45.075ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
44672024-08-22T17:54:45.075ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("6507c79b-9a18-46c7-930d-c98e8662b87f.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("a09ea923-b85c-428d-b289-6936a0c2542c.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("c0ef020c-52c1-4b9a-a085-445fab31264f.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("c0ef020c-52c1-4b9a-a085-445fab31264f.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:3::1))) }], valid_until: Instant { tv_sec: 1873, tv_nsec: 602835446 } })
44682024-08-22T17:54:45.076ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
44692024-08-22T17:54:45.076ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
44702024-08-22T17:54:45.076ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("9488ce0a-6732-4f75-baf9-4ecfa406df91.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("afac2eab-cadf-48d0-94a7-ad562ade88c6.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("afac2eab-cadf-48d0-94a7-ad562ade88c6.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::9))) }], valid_until: Instant { tv_sec: 1873, tv_nsec: 603682940 } })
44712024-08-22T17:54:45.078ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:55
44722024-08-22T17:54:45.078ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
44732024-08-22T17:54:45.078ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
44742024-08-22T17:54:45.078ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2024-08-22 17:44:45.073922790 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
44752024-08-22T17:54:45.078ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 4.703564ms iteration = 6 status = {"expiration":"2024-08-22T17:44:45.073922790Z","pruned":[]}
44762024-08-22T17:54:45.078ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 4.641024ms iteration = 11 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
44772024-08-22T17:54:45.078ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
44782024-08-22T17:54:45.078ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
44792024-08-22T17:54:45.078ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
44802024-08-22T17:54:45.079ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
44812024-08-22T17:54:45.080ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
44822024-08-22T17:54:45.080ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } dns_group = internal
44832024-08-22T17:54:45.080ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 6.201792ms iteration = 6 status = {"addresses":["[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:3::1]:5353"]}
44842024-08-22T17:54:45.080ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
44852024-08-22T17:54:45.081ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
44862024-08-22T17:54:45.081ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 7.418419ms iteration = 6 status = {"addresses":["[fd00:1122:3344:101::8]:5353","[fd00:1122:3344:101::9]:5353"]}
44872024-08-22T17:54:45.081ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 7.203797ms iteration = 6 status = {"errors":[],"found_port_ok":[]}
44882024-08-22T17:54:45.081ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-c961ede4-7532-4ca1-95a5-690b4d7c9b5a.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-c961ede4-7532-4ca1-95a5-690b4d7c9b5a.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1873, tv_nsec: 608376294 } })
44892024-08-22T17:54:45.081ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
44902024-08-22T17:54:45.082ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1050
44912024-08-22T17:54:45.082ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
44922024-08-22T17:54:45.082ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
44932024-08-22T17:54:45.084ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
44942024-08-22T17:54:45.085ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
44952024-08-22T17:54:45.085ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
44962024-08-22T17:54:45.085ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
44972024-08-22T17:54:45.085ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
44982024-08-22T17:54:45.085ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
44992024-08-22T17:54:45.085ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
45002024-08-22T17:54:45.085ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
45012024-08-22T17:54:45.086ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
45022024-08-22T17:54:45.086ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
45032024-08-22T17:54:45.086ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
45042024-08-22T17:54:45.088ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45052024-08-22T17:54:45.092ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
45062024-08-22T17:54:45.092ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
45072024-08-22T17:54:45.092ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
45082024-08-22T17:54:45.094ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45092024-08-22T17:54:45.097ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-c961ede4-7532-4ca1-95a5-690b4d7c9b5a.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-c961ede4-7532-4ca1-95a5-690b4d7c9b5a.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1873, tv_nsec: 624188606 } })
45102024-08-22T17:54:45.097ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1050
45112024-08-22T17:54:45.098ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
45122024-08-22T17:54:45.098ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
45132024-08-22T17:54:45.098ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
45142024-08-22T17:54:45.098ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
45152024-08-22T17:54:45.098ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
45162024-08-22T17:54:45.098ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-c961ede4-7532-4ca1-95a5-690b4d7c9b5a.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-c961ede4-7532-4ca1-95a5-690b4d7c9b5a.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1873, tv_nsec: 625697004 } })
45172024-08-22T17:54:45.099ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45182024-08-22T17:54:45.100ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1050 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561
45192024-08-22T17:54:45.101ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
45202024-08-22T17:54:45.101ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
45212024-08-22T17:54:45.101ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
45222024-08-22T17:54:45.101ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
45232024-08-22T17:54:45.102ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
45242024-08-22T17:54:45.102ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
45252024-08-22T17:54:45.102ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
45262024-08-22T17:54:45.105ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
45272024-08-22T17:54:45.105ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
45282024-08-22T17:54:45.106ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
45292024-08-22T17:54:45.106ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
45302024-08-22T17:54:45.126ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
45312024-08-22T17:54:45.127ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
45322024-08-22T17:54:45.130ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
45332024-08-22T17:54:45.131ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
45342024-08-22T17:54:45.134ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
45352024-08-22T17:54:45.134ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
45362024-08-22T17:54:45.135ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 60.569942ms iteration = 31 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
45372024-08-22T17:54:45.138ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
45382024-08-22T17:54:45.139ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
45392024-08-22T17:54:45.141ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
45402024-08-22T17:54:45.141ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
45412024-08-22T17:54:45.142ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45422024-08-22T17:54:45.144ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:55
45432024-08-22T17:54:45.144ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:220 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
45442024-08-22T17:54:45.144ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 70.522004ms iteration = 11 status = {"destroyed_active_vmms":0,"error":null,"saga_start_failures":0,"sagas_completed":0,"sagas_failed":0,"sagas_started":0,"terminated_active_migrations":0}
45452024-08-22T17:54:45.145ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
45462024-08-22T17:54:45.145ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
45472024-08-22T17:54:45.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
45482024-08-22T17:54:45.146ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45492024-08-22T17:54:45.147ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45502024-08-22T17:54:45.148ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45512024-08-22T17:54:45.150ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
45522024-08-22T17:54:45.151ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
45532024-08-22T17:54:45.151ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0
45542024-08-22T17:54:45.151ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:189 found = 0 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0
45552024-08-22T17:54:45.151ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
45562024-08-22T17:54:45.151ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 77.135128ms iteration = 6 status = {"error":null,"error_count":0,"found":0,"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0}
45572024-08-22T17:54:45.151ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
45582024-08-22T17:54:45.151ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
45592024-08-22T17:54:45.152ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
45602024-08-22T17:54:45.152ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
45612024-08-22T17:54:45.152ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
45622024-08-22T17:54:45.152ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
45632024-08-22T17:54:45.152ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
45642024-08-22T17:54:45.152ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
45652024-08-22T17:54:45.152ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
45662024-08-22T17:54:45.153ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
45672024-08-22T17:54:45.154ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 80.215165ms iteration = 11 status = {"region_replacement_started_err":0,"region_replacement_started_ok":0}
45682024-08-22T17:54:45.154ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 79.020989ms iteration = 11 status = {"errors":[],"garbage_collect_requested":[]}
45692024-08-22T17:54:45.155ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
45702024-08-22T17:54:45.155ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45712024-08-22T17:54:45.155ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
45722024-08-22T17:54:45.156ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45732024-08-22T17:54:45.157ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
45742024-08-22T17:54:45.157ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
45752024-08-22T17:54:45.160ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
45762024-08-22T17:54:45.160ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
45772024-08-22T17:54:45.160ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 147bdc10-344c-4ee8-a219-603cfc66e6d2, lookup_type: ById(147bdc10-344c-4ee8-a219-603cfc66e6d2) } result = Ok(())
45782024-08-22T17:54:45.160ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
45792024-08-22T17:54:45.161ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
45802024-08-22T17:54:45.161ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
45812024-08-22T17:54:45.161ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
45822024-08-22T17:54:45.161ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45832024-08-22T17:54:45.161ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
45842024-08-22T17:54:45.161ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
45852024-08-22T17:54:45.161ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
45862024-08-22T17:54:45.163ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45872024-08-22T17:54:45.164ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
45882024-08-22T17:54:45.164ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
45892024-08-22T17:54:45.165ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
45902024-08-22T17:54:45.167ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
45912024-08-22T17:54:45.167ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
45922024-08-22T17:54:45.168ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: c961ede4-7532-4ca1-95a5-690b4d7c9b5a, lookup_type: ById(c961ede4-7532-4ca1-95a5-690b4d7c9b5a) } result = Ok(())
45932024-08-22T17:54:45.168ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
45942024-08-22T17:54:45.168ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
45952024-08-22T17:54:45.170ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45962024-08-22T17:54:45.173ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
45972024-08-22T17:54:45.173ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
45982024-08-22T17:54:45.173ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 99.086032ms iteration = 11 status = 0
45992024-08-22T17:54:45.173ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
46002024-08-22T17:54:45.178ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
46012024-08-22T17:54:45.179ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 147bdc10-344c-4ee8-a219-603cfc66e6d2 original_time_created = 2024-08-22 17:49:18.445237 UTC target_id = 147bdc10-344c-4ee8-a219-603cfc66e6d2 time_created = 2024-08-22 17:49:18.445237 UTC
46022024-08-22T17:54:45.179ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 105.901478ms iteration = 31 status = {"status":"target blueprint unchanged","target_id":"147bdc10-344c-4ee8-a219-603cfc66e6d2","time_created":"2024-08-22T17:49:18.445237Z"}
46032024-08-22T17:54:45.180ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
46042024-08-22T17:54:45.180ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
46052024-08-22T17:54:45.181ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 107.004973ms iteration = 11 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
46062024-08-22T17:54:45.181ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
46072024-08-22T17:54:45.183ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
46082024-08-22T17:54:45.183ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
46092024-08-22T17:54:45.183ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-08-22T17:49:43.060798Z, time_modified: 2024-08-22T17:49:43.060798Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 10d39b39-43d3-41db-b622-3ef3e09e2dad, db_silo: Silo { identity: SiloIdentity { id: 10d39b39-43d3-41db-b622-3ef3e09e2dad, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-08-22T17:49:58.442383Z, time_modified: 2024-08-22T17:49:58.442383Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "852177ddd7117113b778290a63910551c7b25a6718e9c8fda4590027c9443fee" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 10d39b39-43d3-41db-b622-3ef3e09e2dad, db_silo: Silo { identity: SiloIdentity { id: 10d39b39-43d3-41db-b622-3ef3e09e2dad, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-08-22T17:49:58.442383Z, time_modified: 2024-08-22T17:49:58.442383Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "852177ddd7117113b778290a63910551c7b25a6718e9c8fda4590027c9443fee" }] }) }
46102024-08-22T17:54:45.183ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 109.704158ms iteration = 6 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"10d39b39-43d3-41db-b622-3ef3e09e2dad","tls_certs":["852177ddd7117113b778290a63910551c7b25a6718e9c8fda4590027c9443fee"]}},"default_endpoint":{"silo_id":"10d39b39-43d3-41db-b622-3ef3e09e2dad","tls_certs":["852177ddd7117113b778290a63910551c7b25a6718e9c8fda4590027c9443fee"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
46112024-08-22T17:54:45.185ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
46122024-08-22T17:54:45.186ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
46132024-08-22T17:54:45.187ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
46142024-08-22T17:54:45.188ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
46152024-08-22T17:54:45.190ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
46162024-08-22T17:54:45.190ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
46172024-08-22T17:54:45.192ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
46182024-08-22T17:54:45.192ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
46192024-08-22T17:54:45.193ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
46202024-08-22T17:54:45.193ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:140 found = 0 not_ready_to_be_deleted = 0
46212024-08-22T17:54:45.193ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 119.369097ms iteration = 6 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
46222024-08-22T17:54:45.193ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
46232024-08-22T17:54:45.198ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
46242024-08-22T17:54:45.198ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
46252024-08-22T17:54:45.198ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
46262024-08-22T17:54:45.200ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
46272024-08-22T17:54:45.200ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
46282024-08-22T17:54:45.202ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
46292024-08-22T17:54:45.202ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
46302024-08-22T17:54:45.203ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
46312024-08-22T17:54:45.203ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
46322024-08-22T17:54:45.203ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
46332024-08-22T17:54:45.203ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
46342024-08-22T17:54:45.203ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
46352024-08-22T17:54:45.203ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
46362024-08-22T17:54:45.205ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
46372024-08-22T17:54:45.205ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
46382024-08-22T17:54:45.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
46392024-08-22T17:54:45.206ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
46402024-08-22T17:54:45.206ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
46412024-08-22T17:54:45.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
46422024-08-22T17:54:45.207ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:454 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
46432024-08-22T17:54:45.207ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 133.686871ms iteration = 11 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
46442024-08-22T17:54:45.207ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
46452024-08-22T17:54:45.208ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
46462024-08-22T17:54:45.208ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
46472024-08-22T17:54:45.208ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
46482024-08-22T17:54:45.208ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
46492024-08-22T17:54:45.208ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
46502024-08-22T17:54:45.208ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
46512024-08-22T17:54:45.208ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
46522024-08-22T17:54:45.208ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
46532024-08-22T17:54:45.208ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
46542024-08-22T17:54:45.208ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
46552024-08-22T17:54:45.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
46562024-08-22T17:54:45.216ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
46572024-08-22T17:54:45.227ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1058 zone_address = fd00:1122:3344:101::2
46582024-08-22T17:54:45.227ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
46592024-08-22T17:54:45.227ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "a63054bc-373d-4f90-9e28-3212031b4c02", "content-length": "26", "date": "Thu, 22 Aug 2024 17:54:44 GMT"} })
46602024-08-22T17:54:45.227ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1063 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
46612024-08-22T17:54:45.229ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1095 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
46622024-08-22T17:54:45.242ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
46632024-08-22T17:54:45.244ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
46642024-08-22T17:54:45.244ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
46652024-08-22T17:54:45.244ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
46662024-08-22T17:54:45.263ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
46672024-08-22T17:54:45.284ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 210.605508ms iteration = 12 status = {"physical_disks_added":0}
46682024-08-22T17:54:45.390ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1058 zone_address = fd00:1122:3344:101::2
46692024-08-22T17:54:45.390ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
46702024-08-22T17:54:45.390ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "48e6784b-0e1a-4b41-8d57-653b1fc3eb38", "content-length": "26", "date": "Thu, 22 Aug 2024 17:54:44 GMT"} })
46712024-08-22T17:54:45.390ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1063 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
46722024-08-22T17:54:45.392ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1095 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
46732024-08-22T17:54:45.395ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1058 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 zone_address = fd00:1122:3344:101::2
46742024-08-22T17:54:45.395ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
46752024-08-22T17:54:45.397ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "2c44b717-de71-4586-ba00-c0f582dc4014", "content-length": "26", "date": "Thu, 22 Aug 2024 17:54:44 GMT"} })
46762024-08-22T17:54:45.397ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1063 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
46772024-08-22T17:54:45.398ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1095 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561
46782024-08-22T17:54:45.409ZDEBGnexus (BackgroundTasks): client request SledAgent = c961ede4-7532-4ca1-95a5-690b4d7c9b5a background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
46792024-08-22T17:54:45.409ZDEBGnexus (BackgroundTasks): client response SledAgent = c961ede4-7532-4ca1-95a5-690b4d7c9b5a background_task = vpc_route_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/vpc-routes", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "25451827-dd72-4118-9a6a-d0ca1a20ddf1", "content-length": "372", "date": "Thu, 22 Aug 2024 17:54:45 GMT"} })
46802024-08-22T17:54:45.409ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
46812024-08-22T17:54:45.409ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
46822024-08-22T17:54:45.410ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
46832024-08-22T17:54:45.411ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
46842024-08-22T17:54:45.411ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
46852024-08-22T17:54:45.412ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
46862024-08-22T17:54:45.412ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = d6f65b4e-2e87-480a-af21-a69b4520828c dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
46872024-08-22T17:54:45.412ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
46882024-08-22T17:54:45.412ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
46892024-08-22T17:54:45.412ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
46902024-08-22T17:54:45.412ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
46912024-08-22T17:54:45.413ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
46922024-08-22T17:54:45.413ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
46932024-08-22T17:54:45.414ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
46942024-08-22T17:54:45.414ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
46952024-08-22T17:54:45.414ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
46962024-08-22T17:54:45.414ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
46972024-08-22T17:54:45.414ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
46982024-08-22T17:54:45.414ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
46992024-08-22T17:54:45.415ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
47002024-08-22T17:54:45.416ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47012024-08-22T17:54:45.417ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47022024-08-22T17:54:45.418ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
47032024-08-22T17:54:45.418ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
47042024-08-22T17:54:45.419ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
47052024-08-22T17:54:45.420ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
47062024-08-22T17:54:45.420ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
47072024-08-22T17:54:45.420ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
47082024-08-22T17:54:45.422ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
47092024-08-22T17:54:45.422ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
47102024-08-22T17:54:45.422ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
47112024-08-22T17:54:45.423ZDEBGnexus (BackgroundTasks): client request SledAgent = c961ede4-7532-4ca1-95a5-690b4d7c9b5a background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
47122024-08-22T17:54:45.424ZDEBGnexus (BackgroundTasks): client response SledAgent = c961ede4-7532-4ca1-95a5-690b4d7c9b5a background_task = vpc_route_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/vpc-routes", query: None, fragment: None }, status: 204, headers: {"x-request-id": "ed0e4807-a0a7-4ad2-b057-3749b8534fa2", "date": "Thu, 22 Aug 2024 17:54:45 GMT"} })
47132024-08-22T17:54:45.425ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = d6f65b4e-2e87-480a-af21-a69b4520828c dns_zone_name = control-plane.oxide.internal found_so_far = 50 version = 1
47142024-08-22T17:54:45.425ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
47152024-08-22T17:54:45.425ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-08-22 17:49:57.683168 UTC dns_group = internal generation = 1
47162024-08-22T17:54:45.425ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 351.239725ms iteration = 6 status = {"generation":1}
47172024-08-22T17:54:45.426ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 352.15913ms iteration = 11 status = {}
47182024-08-22T17:54:45.442ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
47192024-08-22T17:54:45.442ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 7f16ce95-fbde-4896-9a22-7f61d8d01c5d dns_zone_name = oxide.test found_so_far = 0 version = 2
47202024-08-22T17:54:45.442ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
47212024-08-22T17:54:45.443ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
47222024-08-22T17:54:45.449ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
47232024-08-22T17:54:45.449ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
47242024-08-22T17:54:45.482ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47252024-08-22T17:54:45.482ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
47262024-08-22T17:54:45.482ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
47272024-08-22T17:54:45.484ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
47282024-08-22T17:54:45.484ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
47292024-08-22T17:54:45.484ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
47302024-08-22T17:54:45.487ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 7f16ce95-fbde-4896-9a22-7f61d8d01c5d dns_zone_name = oxide.test found_so_far = 1 version = 2
47312024-08-22T17:54:45.487ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
47322024-08-22T17:54:45.487ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-08-22 17:49:58.502032 UTC dns_group = external generation = 2
47332024-08-22T17:54:45.487ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 413.072945ms iteration = 6 status = {"generation":2}
47342024-08-22T17:54:45.518ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47352024-08-22T17:54:45.523ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
47362024-08-22T17:54:45.523ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
47372024-08-22T17:54:45.524ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
47382024-08-22T17:54:45.530ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
47392024-08-22T17:54:45.531ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
47402024-08-22T17:54:45.533ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
47412024-08-22T17:54:45.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
47422024-08-22T17:54:45.535ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
47432024-08-22T17:54:45.535ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
47442024-08-22T17:54:45.538ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
47452024-08-22T17:54:45.538ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
47462024-08-22T17:54:45.540ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
47472024-08-22T17:54:45.540ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
47482024-08-22T17:54:45.542ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
47492024-08-22T17:54:45.542ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
47502024-08-22T17:54:45.544ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
47512024-08-22T17:54:45.544ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
47522024-08-22T17:54:45.545ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47532024-08-22T17:54:45.551ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
47542024-08-22T17:54:45.551ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
47552024-08-22T17:54:45.551ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
47562024-08-22T17:54:45.557ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 483.82346ms iteration = 11 status = {}
47572024-08-22T17:54:45.568ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
47582024-08-22T17:54:45.568ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "0f08dde9-6645-4d62-aa7e-160defd4c2c7", "content-length": "1", "date": "Thu, 22 Aug 2024 17:54:44 GMT"} })
47592024-08-22T17:54:45.570ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
47602024-08-22T17:54:45.570ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
47612024-08-22T17:54:45.575ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
47622024-08-22T17:54:45.577ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::8)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "8bd4ee5a-870c-4b96-b38c-432de6b1baf8", "date": "Thu, 22 Aug 2024 17:54:45 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
47632024-08-22T17:54:45.577ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
47642024-08-22T17:54:45.579ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::9)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "6d5eaba2-3fb8-4150-a445-6b991aafbe82", "date": "Thu, 22 Aug 2024 17:54:45 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
47652024-08-22T17:54:45.579ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
47662024-08-22T17:54:45.583ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
47672024-08-22T17:54:45.583ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 509.782854ms iteration = 8 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
47682024-08-22T17:54:45.584ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
47692024-08-22T17:54:45.585ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:2::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "7d999535-8518-4600-a47c-a3bce5d97077", "date": "Thu, 22 Aug 2024 17:54:45 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
47702024-08-22T17:54:45.585ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
47712024-08-22T17:54:45.587ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:1::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "ca8c9416-9f02-4601-99d3-49bb5ebf1cec", "date": "Thu, 22 Aug 2024 17:54:45 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
47722024-08-22T17:54:45.587ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
47732024-08-22T17:54:45.588ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:3::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "872cf9ad-2b7a-4368-a5d4-e6c9a5cfea26", "date": "Thu, 22 Aug 2024 17:54:45 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
47742024-08-22T17:54:45.589ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
47752024-08-22T17:54:45.590ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
47762024-08-22T17:54:45.590ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 516.816241ms iteration = 8 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
47772024-08-22T17:54:45.594ZDEBGnexus (BackgroundTasks): client request SledAgent = c961ede4-7532-4ca1-95a5-690b4d7c9b5a background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
47782024-08-22T17:54:45.594ZDEBGnexus (BackgroundTasks): client response SledAgent = c961ede4-7532-4ca1-95a5-690b4d7c9b5a background_task = v2p_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/v2p", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "c2e2fbd4-fd81-47d7-80cb-8068e47436ae", "content-length": "691", "date": "Thu, 22 Aug 2024 17:54:45 GMT"} })
47792024-08-22T17:54:45.594ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = buskin
47802024-08-22T17:54:45.594ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = buskin
47812024-08-22T17:54:45.594ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = buskin
47822024-08-22T17:54:45.595ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
47832024-08-22T17:54:45.595ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
47842024-08-22T17:54:45.596ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 521.954137ms iteration = 11 status = {}
47852024-08-22T17:54:45.596ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 522.53325ms iteration = 11 status = null
47862024-08-22T17:54:45.596ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
47872024-08-22T17:54:45.597ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
47882024-08-22T17:54:45.598ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47892024-08-22T17:54:45.601ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
47902024-08-22T17:54:45.601ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
47912024-08-22T17:54:45.602ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
47922024-08-22T17:54:45.608ZDEBGnexus (service-firewall-rule-progator): resolved names for firewall rules background_task = service_firewall_rule_propagation instance_interfaces = {} subnet_interfaces = {Name("nexus"): [NetworkInterface { id: 09f89158-2ea8-4bfd-911c-3bf3ffef2aa4, kind: Service { id: 1fa445cf-528e-4b77-85ed-445f0a227ac1 }, name: Name("nexus-1fa445cf-528e-4b77-85ed-445f0a227ac1"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 205, 230])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 5c255d57-c119-4de4-b8a2-7112b0b74875, kind: Service { id: eaa260db-2b19-4ae1-97e8-5735c9f5d955 }, name: Name("nexus-eaa260db-2b19-4ae1-97e8-5735c9f5d955"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 249, 182])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 5d88cb36-556b-42c6-a63f-36eaaaebfa41, kind: Service { id: 21688317-2226-405d-92aa-f8b1720a4f0c }, name: Name("nexus-21688317-2226-405d-92aa-f8b1720a4f0c"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 138, 172])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], Name("external-dns"): [NetworkInterface { id: 88350ddb-4ac1-44fa-af24-3dfceef6dec3, kind: Service { id: afac2eab-cadf-48d0-94a7-ad562ade88c6 }, name: Name("external-dns-afac2eab-cadf-48d0-94a7-ad562ade88c6"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 169, 222])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: bc0e4e8b-293a-4626-9868-98cd4702f955, kind: Service { id: 9488ce0a-6732-4f75-baf9-4ecfa406df91 }, name: Name("external-dns-9488ce0a-6732-4f75-baf9-4ecfa406df91"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 223, 41])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }]} subnet_networks = {Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })], Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })]} vpc_interfaces = {}
47932024-08-22T17:54:45.608ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
47942024-08-22T17:54:45.608ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
47952024-08-22T17:54:45.610ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47962024-08-22T17:54:45.611ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Fleet result = Ok(())
47972024-08-22T17:54:45.611ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
47982024-08-22T17:54:45.611ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
47992024-08-22T17:54:45.614ZDEBGnexus (service-firewall-rule-progator): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified. background_task = service_firewall_rule_propagation
48002024-08-22T17:54:45.617ZDEBGnexus (service-firewall-rule-progator): resolved firewall rules for sled agents background_task = service_firewall_rule_propagation sled_agent_rules = [ResolvedVpcFirewallRule { action: Allow, direction: Inbound, filter_hosts: None, filter_ports: Some([L4PortRange("53")]), filter_protocols: Some([Udp]), priority: 65534, status: Enabled, targets: [NetworkInterface { id: 88350ddb-4ac1-44fa-af24-3dfceef6dec3, kind: Service { id: afac2eab-cadf-48d0-94a7-ad562ade88c6 }, name: Name("external-dns-afac2eab-cadf-48d0-94a7-ad562ade88c6"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 169, 222])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: bc0e4e8b-293a-4626-9868-98cd4702f955, kind: Service { id: 9488ce0a-6732-4f75-baf9-4ecfa406df91 }, name: Name("external-dns-9488ce0a-6732-4f75-baf9-4ecfa406df91"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 223, 41])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }] }, ResolvedVpcFirewallRule { action: Allow, direction: Inbound, filter_hosts: None, filter_ports: Some([L4PortRange("80"), L4PortRange("443")]), filter_protocols: Some([Tcp]), priority: 65534, status: Enabled, targets: [NetworkInterface { id: 09f89158-2ea8-4bfd-911c-3bf3ffef2aa4, kind: Service { id: 1fa445cf-528e-4b77-85ed-445f0a227ac1 }, name: Name("nexus-1fa445cf-528e-4b77-85ed-445f0a227ac1"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 205, 230])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 5c255d57-c119-4de4-b8a2-7112b0b74875, kind: Service { id: eaa260db-2b19-4ae1-97e8-5735c9f5d955 }, name: Name("nexus-eaa260db-2b19-4ae1-97e8-5735c9f5d955"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 249, 182])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 5d88cb36-556b-42c6-a63f-36eaaaebfa41, kind: Service { id: 21688317-2226-405d-92aa-f8b1720a4f0c }, name: Name("nexus-21688317-2226-405d-92aa-f8b1720a4f0c"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 138, 172])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }] }]
48012024-08-22T17:54:45.617ZDEBGnexus (service-firewall-rule-progator): resolved 2 rules for sleds background_task = service_firewall_rule_propagation
48022024-08-22T17:54:45.644ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
48032024-08-22T17:54:45.644ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
48042024-08-22T17:54:45.647ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:147 port = SwitchPort { id: c772d372-7448-4ce0-b8d1-efa645d18ed1, rack_id: c66d93a6-e2c3-4b19-9564-3b58bad50561, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(9ff05247-84aa-47d8-8619-ce0a8dbbaf7c) } rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 switch_location = Switch0
48052024-08-22T17:54:45.647ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
48062024-08-22T17:54:45.647ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
48072024-08-22T17:54:45.657ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
48082024-08-22T17:54:45.658ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/port/qsfp0/settings", query: Some("tag=omicron"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "19391ff1-16be-426e-bf56-1445e95f2920", "content-length": "121", "date": "Thu, 22 Aug 2024 17:54:44 GMT"} })
48092024-08-22T17:54:45.658ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1714 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G } }} } switch_port_id = "qsfp0"
48102024-08-22T17:54:45.658ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1741 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 9ff05247-84aa-47d8-8619-ce0a8dbbaf7c
48112024-08-22T17:54:45.658ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:379 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
48122024-08-22T17:54:45.658ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
48132024-08-22T17:54:45.658ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:392 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 routes = {}
48142024-08-22T17:54:45.658ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:398 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 routes = {}
48152024-08-22T17:54:45.658ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:418 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561
48162024-08-22T17:54:45.658ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
48172024-08-22T17:54:45.658ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
48182024-08-22T17:54:45.660ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
48192024-08-22T17:54:45.661ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv4", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "c20aa874-2d3d-4d62-a64b-e06d6a540563", "content-length": "2", "date": "Thu, 22 Aug 2024 17:54:44 GMT"} })
48202024-08-22T17:54:45.661ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
48212024-08-22T17:54:45.661ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "41c19aeb-08a5-4114-9600-fa6bc6fa7045", "content-length": "52", "date": "Thu, 22 Aug 2024 17:54:44 GMT"} })
48222024-08-22T17:54:45.661ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0" }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:467 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 switch_location = Switch0
48232024-08-22T17:54:45.661ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
48242024-08-22T17:54:45.736ZDEBGnexus (service-firewall-rule-progator): resolved sleds for vpc oxide-services background_task = service_firewall_rule_propagation vpc_to_sled = [Sled { identity: SledIdentity { id: c961ede4-7532-4ca1-95a5-690b4d7c9b5a, time_created: 2024-08-22T17:49:54.055103Z, time_modified: 2024-08-22T17:49:54.055103Z }, time_deleted: None, rcgen: Generation(Generation(19)), rack_id: c66d93a6-e2c3-4b19-9564-3b58bad50561, is_scrimlet: true, serial_number: "buskin", part_number: "i86pc", revision: SqlU32(0), usable_hardware_threads: SqlU32(48), usable_physical_ram: ByteCount(ByteCount(102952026112)), reservoir_size: ByteCount(ByteCount(51474595840)), ip: fd00:1122:3344:101::1, port: SqlU16(12345), last_used_address: fd00:1122:3344:101::ffff, policy: InService, state: Active, sled_agent_gen: Generation(Generation(1)) }]
48252024-08-22T17:54:45.736ZDEBGnexus (service-firewall-rule-progator): sending firewall rules to sled agents background_task = service_firewall_rule_propagation
48262024-08-22T17:54:45.736ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
48272024-08-22T17:54:45.736ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
48282024-08-22T17:54:45.738ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
48292024-08-22T17:54:45.738ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
48302024-08-22T17:54:45.773ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
48312024-08-22T17:54:45.775ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Sled { parent: Fleet, key: c961ede4-7532-4ca1-95a5-690b4d7c9b5a, lookup_type: ById(c961ede4-7532-4ca1-95a5-690b4d7c9b5a) } result = Ok(())
48322024-08-22T17:54:45.817ZDEBGnexus (service-firewall-rule-progator): client request SledAgent = c961ede4-7532-4ca1-95a5-690b4d7c9b5a background_task = service_firewall_rule_propagation body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules
48332024-08-22T17:54:45.818ZDEBGnexus (service-firewall-rule-progator): client response SledAgent = c961ede4-7532-4ca1-95a5-690b4d7c9b5a background_task = service_firewall_rule_propagation result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/vpc/001de000-074c-4000-8000-000000000000/firewall/rules", query: None, fragment: None }, status: 204, headers: {"x-request-id": "27480cd9-ea40-4178-b1fa-317c5de5214d", "date": "Thu, 22 Aug 2024 17:54:45 GMT"} })
48342024-08-22T17:54:45.820ZINFOnexus (service-firewall-rule-progator): updated firewall rules on 1 sleds background_task = service_firewall_rule_propagation file = nexus/networking/src/firewall_rules.rs:496
48352024-08-22T17:54:45.820ZDEBGnexus (service-firewall-rule-progator): successfully propagated service firewall rules background_task = service_firewall_rule_propagation elapsed = 746.466148ms
48362024-08-22T17:54:45.820ZDEBGnexus (BackgroundTasks): activation complete background_task = service_firewall_rule_propagation elapsed = 746.475718ms iteration = 2 status = {}
48372024-08-22T17:54:45.823ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/switch-ports", query: None, fragment: None }, status: 204, headers: {"x-request-id": "1cb052e8-98c9-47df-bed7-cafaadeb1464", "date": "Thu, 22 Aug 2024 17:54:45 GMT"} })
48382024-08-22T17:54:45.823ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:770 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 switch = Switch0
48392024-08-22T17:54:45.823ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
48402024-08-22T17:54:45.824ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/network-bootstore-config", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "618f75ba-d4a0-4409-bc56-af58b9965980", "content-length": "667", "date": "Thu, 22 Aug 2024 17:54:45 GMT"} })
48412024-08-22T17:54:45.824ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
48422024-08-22T17:54:45.824ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
48432024-08-22T17:54:45.827ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
48442024-08-22T17:54:45.827ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
48452024-08-22T17:54:45.830ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
48462024-08-22T17:54:45.830ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
48472024-08-22T17:54:45.832ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
48482024-08-22T17:54:45.833ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
48492024-08-22T17:54:45.839ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 765.580827ms iteration = 11 status = {}
48502024-08-22T17:54:46.007ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 214 local_addr = [fd00:1122:3344:101::c]:56994 method = GET remote_addr = [fd00:1122:3344:101::d]:40670 req_id = 9c1ade52-9021-454a-a960-27740559bd40 response_code = 200 uri = /eaa260db-2b19-4ae1-97e8-5735c9f5d955
48512024-08-22T17:54:55.074ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 32 reason = Timeout
48522024-08-22T17:54:55.074ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 32 reason = Timeout
48532024-08-22T17:54:55.074ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
48542024-08-22T17:54:55.074ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
48552024-08-22T17:54:55.074ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
48562024-08-22T17:54:55.074ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
48572024-08-22T17:54:55.102ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
48582024-08-22T17:54:55.102ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
48592024-08-22T17:54:55.102ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
48602024-08-22T17:54:55.105ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
48612024-08-22T17:54:55.105ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
48622024-08-22T17:54:55.105ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
48632024-08-22T17:54:55.114ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
48642024-08-22T17:54:55.114ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
48652024-08-22T17:54:55.115ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
48662024-08-22T17:54:55.118ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 147bdc10-344c-4ee8-a219-603cfc66e6d2, lookup_type: ById(147bdc10-344c-4ee8-a219-603cfc66e6d2) } result = Ok(())
48672024-08-22T17:54:55.118ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
48682024-08-22T17:54:55.118ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
48692024-08-22T17:54:55.129ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 147bdc10-344c-4ee8-a219-603cfc66e6d2 original_time_created = 2024-08-22 17:49:18.445237 UTC target_id = 147bdc10-344c-4ee8-a219-603cfc66e6d2 time_created = 2024-08-22 17:49:18.445237 UTC
48702024-08-22T17:54:55.129ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 55.660218ms iteration = 32 status = {"status":"target blueprint unchanged","target_id":"147bdc10-344c-4ee8-a219-603cfc66e6d2","time_created":"2024-08-22T17:49:18.445237Z"}
48712024-08-22T17:54:55.143ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 69.192737ms iteration = 32 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
48722024-08-22T17:54:56.007ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 206 local_addr = [fd00:1122:3344:101::c]:56994 method = GET remote_addr = [fd00:1122:3344:101::d]:40670 req_id = d45ddefe-d916-46a8-bdf2-8c08f9b2048d response_code = 200 uri = /eaa260db-2b19-4ae1-97e8-5735c9f5d955
48732024-08-22T17:55:05.075ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 33 reason = Timeout
48742024-08-22T17:55:05.075ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 33 reason = Timeout
48752024-08-22T17:55:05.075ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
48762024-08-22T17:55:05.075ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
48772024-08-22T17:55:05.075ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
48782024-08-22T17:55:05.075ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
48792024-08-22T17:55:05.076ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
48802024-08-22T17:55:05.079ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
48812024-08-22T17:55:05.079ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
48822024-08-22T17:55:05.079ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
48832024-08-22T17:55:05.080ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
48842024-08-22T17:55:05.080ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
48852024-08-22T17:55:05.082ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
48862024-08-22T17:55:05.085ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 147bdc10-344c-4ee8-a219-603cfc66e6d2, lookup_type: ById(147bdc10-344c-4ee8-a219-603cfc66e6d2) } result = Ok(())
48872024-08-22T17:55:05.085ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
48882024-08-22T17:55:05.085ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
48892024-08-22T17:55:05.093ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 147bdc10-344c-4ee8-a219-603cfc66e6d2 original_time_created = 2024-08-22 17:49:18.445237 UTC target_id = 147bdc10-344c-4ee8-a219-603cfc66e6d2 time_created = 2024-08-22 17:49:18.445237 UTC
48902024-08-22T17:55:05.093ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 18.106344ms iteration = 33 status = {"status":"target blueprint unchanged","target_id":"147bdc10-344c-4ee8-a219-603cfc66e6d2","time_created":"2024-08-22T17:49:18.445237Z"}
48912024-08-22T17:55:05.107ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
48922024-08-22T17:55:05.108ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
48932024-08-22T17:55:05.109ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 34.401208ms iteration = 33 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
48942024-08-22T17:55:06.008ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 236 local_addr = [fd00:1122:3344:101::c]:56994 method = GET remote_addr = [fd00:1122:3344:101::d]:40670 req_id = 6d147d18-cc41-497f-af15-41c18d43bb8a response_code = 200 uri = /eaa260db-2b19-4ae1-97e8-5735c9f5d955
48952024-08-22T17:55:15.074ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 34 reason = Timeout
48962024-08-22T17:55:15.075ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 12 reason = Timeout
48972024-08-22T17:55:15.075ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
48982024-08-22T17:55:15.075ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
48992024-08-22T17:55:15.075ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 12 reason = Timeout
49002024-08-22T17:55:15.075ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 12 reason = Timeout
49012024-08-22T17:55:15.075ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
49022024-08-22T17:55:15.075ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
49032024-08-22T17:55:15.075ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 12 reason = Timeout
49042024-08-22T17:55:15.075ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 12 reason = Timeout
49052024-08-22T17:55:15.075ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
49062024-08-22T17:55:15.075ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 13 reason = Timeout
49072024-08-22T17:55:15.075ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
49082024-08-22T17:55:15.075ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 12 reason = Timeout
49092024-08-22T17:55:15.075ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 12 reason = Timeout
49102024-08-22T17:55:15.075ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
49112024-08-22T17:55:15.075ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 12 reason = Timeout
49122024-08-22T17:55:15.075ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
49132024-08-22T17:55:15.075ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 12 reason = Timeout
49142024-08-22T17:55:15.075ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
49152024-08-22T17:55:15.075ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
49162024-08-22T17:55:15.075ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
49172024-08-22T17:55:15.075ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 12 reason = Timeout
49182024-08-22T17:55:15.075ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
49192024-08-22T17:55:15.075ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 12 reason = Timeout
49202024-08-22T17:55:15.075ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
49212024-08-22T17:55:15.075ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 34 reason = Timeout
49222024-08-22T17:55:15.075ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
49232024-08-22T17:55:15.075ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
49242024-08-22T17:55:15.075ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
49252024-08-22T17:55:15.075ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
49262024-08-22T17:55:15.075ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
49272024-08-22T17:55:15.075ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
49282024-08-22T17:55:15.075ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
49292024-08-22T17:55:15.075ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
49302024-08-22T17:55:15.075ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
49312024-08-22T17:55:15.075ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
49322024-08-22T17:55:15.075ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
49332024-08-22T17:55:15.075ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
49342024-08-22T17:55:15.075ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
49352024-08-22T17:55:15.075ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
49362024-08-22T17:55:15.077ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 12 reason = Timeout
49372024-08-22T17:55:15.077ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
49382024-08-22T17:55:15.077ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
49392024-08-22T17:55:15.077ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
49402024-08-22T17:55:15.078ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
49412024-08-22T17:55:15.078ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.68109ms iteration = 12 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
49422024-08-22T17:55:15.079ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
49432024-08-22T17:55:15.080ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-c961ede4-7532-4ca1-95a5-690b4d7c9b5a.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-c961ede4-7532-4ca1-95a5-690b4d7c9b5a.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1903, tv_nsec: 606686936 } })
49442024-08-22T17:55:15.080ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1050
49452024-08-22T17:55:15.080ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
49462024-08-22T17:55:15.080ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
49472024-08-22T17:55:15.081ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
49482024-08-22T17:55:15.081ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
49492024-08-22T17:55:15.081ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
49502024-08-22T17:55:15.081ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
49512024-08-22T17:55:15.081ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
49522024-08-22T17:55:15.081ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
49532024-08-22T17:55:15.081ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
49542024-08-22T17:55:15.084ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
49552024-08-22T17:55:15.085ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
49562024-08-22T17:55:15.086ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
49572024-08-22T17:55:15.086ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-c961ede4-7532-4ca1-95a5-690b4d7c9b5a.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-c961ede4-7532-4ca1-95a5-690b4d7c9b5a.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1903, tv_nsec: 612778808 } })
49582024-08-22T17:55:15.086ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1050
49592024-08-22T17:55:15.087ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
49602024-08-22T17:55:15.087ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
49612024-08-22T17:55:15.087ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
49622024-08-22T17:55:15.090ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
49632024-08-22T17:55:15.090ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
49642024-08-22T17:55:15.090ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
49652024-08-22T17:55:15.091ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
49662024-08-22T17:55:15.091ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
49672024-08-22T17:55:15.094ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
49682024-08-22T17:55:15.094ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
49692024-08-22T17:55:15.094ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
49702024-08-22T17:55:15.128ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
49712024-08-22T17:55:15.128ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
49722024-08-22T17:55:15.132ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 55.262945ms iteration = 12 status = {"errors":[],"garbage_collect_requested":[]}
49732024-08-22T17:55:15.132ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
49742024-08-22T17:55:15.135ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:55
49752024-08-22T17:55:15.135ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
49762024-08-22T17:55:15.135ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
49772024-08-22T17:55:15.135ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
49782024-08-22T17:55:15.135ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
49792024-08-22T17:55:15.135ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
49802024-08-22T17:55:15.136ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
49812024-08-22T17:55:15.137ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
49822024-08-22T17:55:15.138ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
49832024-08-22T17:55:15.139ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
49842024-08-22T17:55:15.141ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
49852024-08-22T17:55:15.141ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
49862024-08-22T17:55:15.141ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
49872024-08-22T17:55:15.143ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
49882024-08-22T17:55:15.144ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
49892024-08-22T17:55:15.144ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-c961ede4-7532-4ca1-95a5-690b4d7c9b5a.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-c961ede4-7532-4ca1-95a5-690b4d7c9b5a.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1903, tv_nsec: 670925938 } })
49902024-08-22T17:55:15.146ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1050 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561
49912024-08-22T17:55:15.158ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
49922024-08-22T17:55:15.160ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
49932024-08-22T17:55:15.161ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
49942024-08-22T17:55:15.161ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 147bdc10-344c-4ee8-a219-603cfc66e6d2, lookup_type: ById(147bdc10-344c-4ee8-a219-603cfc66e6d2) } result = Ok(())
49952024-08-22T17:55:15.161ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
49962024-08-22T17:55:15.161ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 86.759019ms iteration = 34 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
49972024-08-22T17:55:15.162ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
49982024-08-22T17:55:15.162ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
49992024-08-22T17:55:15.164ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
50002024-08-22T17:55:15.164ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
50012024-08-22T17:55:15.164ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1058 zone_address = fd00:1122:3344:101::2
50022024-08-22T17:55:15.164ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
50032024-08-22T17:55:15.164ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
50042024-08-22T17:55:15.164ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
50052024-08-22T17:55:15.164ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
50062024-08-22T17:55:15.164ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "afaa71d7-cc26-48e9-afba-48ba9ecb76cc", "content-length": "26", "date": "Thu, 22 Aug 2024 17:55:14 GMT"} })
50072024-08-22T17:55:15.164ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1063 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
50082024-08-22T17:55:15.164ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1058 zone_address = fd00:1122:3344:101::2
50092024-08-22T17:55:15.164ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
50102024-08-22T17:55:15.165ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "d9a44b38-045a-4189-8a25-3244608621b3", "content-length": "26", "date": "Thu, 22 Aug 2024 17:55:14 GMT"} })
50112024-08-22T17:55:15.165ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1063 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
50122024-08-22T17:55:15.166ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1095 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
50132024-08-22T17:55:15.167ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1095 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
50142024-08-22T17:55:15.170ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:55
50152024-08-22T17:55:15.170ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:220 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
50162024-08-22T17:55:15.170ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
50172024-08-22T17:55:15.170ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 95.156312ms iteration = 12 status = {"destroyed_active_vmms":0,"error":null,"saga_start_failures":0,"sagas_completed":0,"sagas_failed":0,"sagas_started":0,"terminated_active_migrations":0}
50182024-08-22T17:55:15.173ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: c961ede4-7532-4ca1-95a5-690b4d7c9b5a, lookup_type: ById(c961ede4-7532-4ca1-95a5-690b4d7c9b5a) } result = Ok(())
50192024-08-22T17:55:15.173ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
50202024-08-22T17:55:15.173ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
50212024-08-22T17:55:15.173ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 98.763761ms iteration = 12 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
50222024-08-22T17:55:15.175ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 100.422069ms iteration = 12 status = 0
50232024-08-22T17:55:15.176ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:454 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
50242024-08-22T17:55:15.176ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 101.106732ms iteration = 12 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
50252024-08-22T17:55:15.180ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 147bdc10-344c-4ee8-a219-603cfc66e6d2 original_time_created = 2024-08-22 17:49:18.445237 UTC target_id = 147bdc10-344c-4ee8-a219-603cfc66e6d2 time_created = 2024-08-22 17:49:18.445237 UTC
50262024-08-22T17:55:15.180ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 105.468275ms iteration = 34 status = {"status":"target blueprint unchanged","target_id":"147bdc10-344c-4ee8-a219-603cfc66e6d2","time_created":"2024-08-22T17:49:18.445237Z"}
50272024-08-22T17:55:15.181ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 106.295389ms iteration = 12 status = {"region_replacement_started_err":0,"region_replacement_started_ok":0}
50282024-08-22T17:55:15.202ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
50292024-08-22T17:55:15.203ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
50302024-08-22T17:55:15.253ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 178.011049ms iteration = 13 status = {"physical_disks_added":0}
50312024-08-22T17:55:15.272ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 197.522341ms iteration = 12 status = {}
50322024-08-22T17:55:15.276ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
50332024-08-22T17:55:15.277ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "f7a2718e-70f0-413f-b706-fc4af84ef62a", "content-length": "1", "date": "Thu, 22 Aug 2024 17:55:14 GMT"} })
50342024-08-22T17:55:15.280ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
50352024-08-22T17:55:15.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
50362024-08-22T17:55:15.282ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 206.94203ms iteration = 12 status = null
50372024-08-22T17:55:15.283ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
50382024-08-22T17:55:15.284ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
50392024-08-22T17:55:15.285ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
50402024-08-22T17:55:15.287ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
50412024-08-22T17:55:15.287ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
50422024-08-22T17:55:15.287ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
50432024-08-22T17:55:15.288ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
50442024-08-22T17:55:15.290ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
50452024-08-22T17:55:15.290ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
50462024-08-22T17:55:15.290ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
50472024-08-22T17:55:15.299ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1058 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 zone_address = fd00:1122:3344:101::2
50482024-08-22T17:55:15.299ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
50492024-08-22T17:55:15.299ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "cb005000-cd3b-4f2f-966b-76f7bb92a62b", "content-length": "26", "date": "Thu, 22 Aug 2024 17:55:14 GMT"} })
50502024-08-22T17:55:15.299ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1063 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
50512024-08-22T17:55:15.301ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1095 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561
50522024-08-22T17:55:15.323ZDEBGnexus (BackgroundTasks): client request SledAgent = c961ede4-7532-4ca1-95a5-690b4d7c9b5a background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
50532024-08-22T17:55:15.323ZDEBGnexus (BackgroundTasks): client response SledAgent = c961ede4-7532-4ca1-95a5-690b4d7c9b5a background_task = vpc_route_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/vpc-routes", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "a01b2823-781a-4247-a5e8-e9dcdae496b5", "content-length": "372", "date": "Thu, 22 Aug 2024 17:55:14 GMT"} })
50542024-08-22T17:55:15.323ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
50552024-08-22T17:55:15.324ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
50562024-08-22T17:55:15.327ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
50572024-08-22T17:55:15.327ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
50582024-08-22T17:55:15.331ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
50592024-08-22T17:55:15.331ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
50602024-08-22T17:55:15.334ZDEBGnexus (BackgroundTasks): client request SledAgent = c961ede4-7532-4ca1-95a5-690b4d7c9b5a background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
50612024-08-22T17:55:15.334ZDEBGnexus (BackgroundTasks): client response SledAgent = c961ede4-7532-4ca1-95a5-690b4d7c9b5a background_task = vpc_route_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/vpc-routes", query: None, fragment: None }, status: 204, headers: {"x-request-id": "bbf2abca-cb7b-4edc-9b74-e671984bd05d", "date": "Thu, 22 Aug 2024 17:55:14 GMT"} })
50622024-08-22T17:55:15.336ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 261.13057ms iteration = 12 status = {}
50632024-08-22T17:55:15.384ZDEBGnexus (BackgroundTasks): client request SledAgent = c961ede4-7532-4ca1-95a5-690b4d7c9b5a background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
50642024-08-22T17:55:15.384ZDEBGnexus (BackgroundTasks): client response SledAgent = c961ede4-7532-4ca1-95a5-690b4d7c9b5a background_task = v2p_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/v2p", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "da93babe-47d9-4c07-b507-6897b01729c7", "content-length": "691", "date": "Thu, 22 Aug 2024 17:55:14 GMT"} })
50652024-08-22T17:55:15.384ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = buskin
50662024-08-22T17:55:15.384ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = buskin
50672024-08-22T17:55:15.384ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = buskin
50682024-08-22T17:55:15.386ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 311.943442ms iteration = 12 status = {}
50692024-08-22T17:55:15.414ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
50702024-08-22T17:55:15.414ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
50712024-08-22T17:55:15.415ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:147 port = SwitchPort { id: c772d372-7448-4ce0-b8d1-efa645d18ed1, rack_id: c66d93a6-e2c3-4b19-9564-3b58bad50561, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(9ff05247-84aa-47d8-8619-ce0a8dbbaf7c) } rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 switch_location = Switch0
50722024-08-22T17:55:15.415ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
50732024-08-22T17:55:15.416ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
50742024-08-22T17:55:15.424ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
50752024-08-22T17:55:15.424ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/port/qsfp0/settings", query: Some("tag=omicron"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "23c7dbb8-b57a-47b3-9321-f0ceb8683d84", "content-length": "121", "date": "Thu, 22 Aug 2024 17:55:14 GMT"} })
50762024-08-22T17:55:15.424ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1714 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G } }} } switch_port_id = "qsfp0"
50772024-08-22T17:55:15.424ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1741 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 9ff05247-84aa-47d8-8619-ce0a8dbbaf7c
50782024-08-22T17:55:15.424ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:379 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
50792024-08-22T17:55:15.424ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
50802024-08-22T17:55:15.424ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:392 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 routes = {}
50812024-08-22T17:55:15.424ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:398 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 routes = {}
50822024-08-22T17:55:15.424ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:418 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561
50832024-08-22T17:55:15.424ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
50842024-08-22T17:55:15.425ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
50852024-08-22T17:55:15.426ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
50862024-08-22T17:55:15.426ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv4", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "049743d4-4b92-47ae-a182-e009e1fffe2e", "content-length": "2", "date": "Thu, 22 Aug 2024 17:55:14 GMT"} })
50872024-08-22T17:55:15.426ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
50882024-08-22T17:55:15.427ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "d7bb1881-21a8-4ad0-a93d-c6b6ff6565dc", "content-length": "52", "date": "Thu, 22 Aug 2024 17:55:14 GMT"} })
50892024-08-22T17:55:15.427ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0" }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:467 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 switch_location = Switch0
50902024-08-22T17:55:15.427ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
50912024-08-22T17:55:15.525ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/switch-ports", query: None, fragment: None }, status: 204, headers: {"x-request-id": "44457c74-03ca-4574-8df9-67872c028b25", "date": "Thu, 22 Aug 2024 17:55:14 GMT"} })
50922024-08-22T17:55:15.525ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:770 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 switch = Switch0
50932024-08-22T17:55:15.525ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
50942024-08-22T17:55:15.640ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/network-bootstore-config", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "a888fc07-65ad-4cf6-b699-316d4507c1a9", "content-length": "667", "date": "Thu, 22 Aug 2024 17:55:15 GMT"} })
50952024-08-22T17:55:15.640ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
50962024-08-22T17:55:15.641ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
50972024-08-22T17:55:15.645ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
50982024-08-22T17:55:15.645ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
50992024-08-22T17:55:15.648ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
51002024-08-22T17:55:15.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
51012024-08-22T17:55:15.650ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
51022024-08-22T17:55:15.650ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
51032024-08-22T17:55:15.656ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 581.539296ms iteration = 12 status = {}
51042024-08-22T17:55:16.008ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 241 local_addr = [fd00:1122:3344:101::c]:56994 method = GET remote_addr = [fd00:1122:3344:101::d]:40670 req_id = f30f2634-a24a-40b0-8073-0fccbe94e681 response_code = 200 uri = /eaa260db-2b19-4ae1-97e8-5735c9f5d955
51052024-08-22T17:55:25.074ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 35 reason = Timeout
51062024-08-22T17:55:25.074ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 35 reason = Timeout
51072024-08-22T17:55:25.074ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
51082024-08-22T17:55:25.074ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
51092024-08-22T17:55:25.075ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
51102024-08-22T17:55:25.075ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
51112024-08-22T17:55:25.077ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
51122024-08-22T17:55:25.077ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
51132024-08-22T17:55:25.077ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
51142024-08-22T17:55:25.079ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
51152024-08-22T17:55:25.079ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
51162024-08-22T17:55:25.079ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
51172024-08-22T17:55:25.081ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
51182024-08-22T17:55:25.082ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
51192024-08-22T17:55:25.083ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
51202024-08-22T17:55:25.085ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 147bdc10-344c-4ee8-a219-603cfc66e6d2, lookup_type: ById(147bdc10-344c-4ee8-a219-603cfc66e6d2) } result = Ok(())
51212024-08-22T17:55:25.085ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
51222024-08-22T17:55:25.086ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
51232024-08-22T17:55:25.094ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 147bdc10-344c-4ee8-a219-603cfc66e6d2 original_time_created = 2024-08-22 17:49:18.445237 UTC target_id = 147bdc10-344c-4ee8-a219-603cfc66e6d2 time_created = 2024-08-22 17:49:18.445237 UTC
51242024-08-22T17:55:25.095ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 20.135874ms iteration = 35 status = {"status":"target blueprint unchanged","target_id":"147bdc10-344c-4ee8-a219-603cfc66e6d2","time_created":"2024-08-22T17:49:18.445237Z"}
51252024-08-22T17:55:25.107ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 32.286227ms iteration = 35 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
51262024-08-22T17:55:26.008ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 313 local_addr = [fd00:1122:3344:101::c]:56994 method = GET remote_addr = [fd00:1122:3344:101::d]:40670 req_id = eccb1914-0595-4a13-b344-a960785948b0 response_code = 200 uri = /eaa260db-2b19-4ae1-97e8-5735c9f5d955
51272024-08-22T17:55:35.075ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 36 reason = Timeout
51282024-08-22T17:55:35.075ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 36 reason = Timeout
51292024-08-22T17:55:35.075ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
51302024-08-22T17:55:35.075ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
51312024-08-22T17:55:35.075ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
51322024-08-22T17:55:35.075ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
51332024-08-22T17:55:35.077ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
51342024-08-22T17:55:35.077ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
51352024-08-22T17:55:35.077ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
51362024-08-22T17:55:35.078ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.717969ms iteration = 36 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
51372024-08-22T17:55:35.079ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
51382024-08-22T17:55:35.079ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
51392024-08-22T17:55:35.079ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
51402024-08-22T17:55:35.081ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
51412024-08-22T17:55:35.081ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
51422024-08-22T17:55:35.083ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
51432024-08-22T17:55:35.085ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 147bdc10-344c-4ee8-a219-603cfc66e6d2, lookup_type: ById(147bdc10-344c-4ee8-a219-603cfc66e6d2) } result = Ok(())
51442024-08-22T17:55:35.085ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
51452024-08-22T17:55:35.085ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
51462024-08-22T17:55:35.094ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 147bdc10-344c-4ee8-a219-603cfc66e6d2 original_time_created = 2024-08-22 17:49:18.445237 UTC target_id = 147bdc10-344c-4ee8-a219-603cfc66e6d2 time_created = 2024-08-22 17:49:18.445237 UTC
51472024-08-22T17:55:35.094ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 19.305099ms iteration = 36 status = {"status":"target blueprint unchanged","target_id":"147bdc10-344c-4ee8-a219-603cfc66e6d2","time_created":"2024-08-22T17:49:18.445237Z"}
51482024-08-22T17:55:36.008ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 287 local_addr = [fd00:1122:3344:101::c]:56994 method = GET remote_addr = [fd00:1122:3344:101::d]:40670 req_id = 0d364ab7-69d7-4b55-9d9d-be6949a0e32e response_code = 200 uri = /eaa260db-2b19-4ae1-97e8-5735c9f5d955
51492024-08-22T17:55:45.075ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 7 reason = Timeout
51502024-08-22T17:55:45.075ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 7 reason = Timeout
51512024-08-22T17:55:45.075ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 9 reason = Timeout
51522024-08-22T17:55:45.075ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
51532024-08-22T17:55:45.075ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 9 reason = Timeout
51542024-08-22T17:55:45.075ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 13 reason = Timeout
51552024-08-22T17:55:45.075ZDEBGnexus (BackgroundTasks): activating background_task = crdb_node_id_collector iteration = 4 reason = Timeout
51562024-08-22T17:55:45.075ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 8 reason = Timeout
51572024-08-22T17:55:45.075ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:80 reason = blueprint disabled target_id = 147bdc10-344c-4ee8-a219-603cfc66e6d2
51582024-08-22T17:55:45.075ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
51592024-08-22T17:55:45.075ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
51602024-08-22T17:55:45.075ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 15.96\u{b5}s iteration = 8 status = {"error":"blueprint disabled","target_id":"147bdc10-344c-4ee8-a219-603cfc66e6d2"}
51612024-08-22T17:55:45.075ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 7 reason = Timeout
51622024-08-22T17:55:45.075ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 13 reason = Timeout
51632024-08-22T17:55:45.075ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 7 reason = Timeout
51642024-08-22T17:55:45.075ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:56
51652024-08-22T17:55:45.075ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
51662024-08-22T17:55:45.075ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 37 reason = Timeout
51672024-08-22T17:55:45.075ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 7 reason = Timeout
51682024-08-22T17:55:45.075ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 7 reason = Timeout
51692024-08-22T17:55:45.075ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
51702024-08-22T17:55:45.075ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
51712024-08-22T17:55:45.075ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
51722024-08-22T17:55:45.075ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
51732024-08-22T17:55:45.075ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
51742024-08-22T17:55:45.075ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 13 reason = Timeout
51752024-08-22T17:55:45.075ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 13 reason = Timeout
51762024-08-22T17:55:45.075ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
51772024-08-22T17:55:45.075ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
51782024-08-22T17:55:45.075ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 37 reason = Timeout
51792024-08-22T17:55:45.075ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
51802024-08-22T17:55:45.075ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 13 reason = Timeout
51812024-08-22T17:55:45.075ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 13 reason = Timeout
51822024-08-22T17:55:45.075ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
51832024-08-22T17:55:45.075ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 13 reason = Timeout
51842024-08-22T17:55:45.075ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 13 reason = Timeout
51852024-08-22T17:55:45.075ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
51862024-08-22T17:55:45.075ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
51872024-08-22T17:55:45.075ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
51882024-08-22T17:55:45.075ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 14 reason = Timeout
51892024-08-22T17:55:45.075ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
51902024-08-22T17:55:45.075ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 7 reason = Timeout
51912024-08-22T17:55:45.075ZINFOnexus (BackgroundTasks): Abandoned VMM reaper running background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:76
51922024-08-22T17:55:45.075ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
51932024-08-22T17:55:45.075ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 7 reason = Timeout
51942024-08-22T17:55:45.075ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
51952024-08-22T17:55:45.075ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 7 reason = Timeout
51962024-08-22T17:55:45.075ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2024-08-22 17:45:45.075666136 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
51972024-08-22T17:55:45.075ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
51982024-08-22T17:55:45.075ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 13 reason = Timeout
51992024-08-22T17:55:45.075ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
52002024-08-22T17:55:45.075ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 13 reason = Timeout
52012024-08-22T17:55:45.075ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
52022024-08-22T17:55:45.075ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
52032024-08-22T17:55:45.075ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 13 reason = Timeout
52042024-08-22T17:55:45.075ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
52052024-08-22T17:55:45.075ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
52062024-08-22T17:55:45.076ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
52072024-08-22T17:55:45.076ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
52082024-08-22T17:55:45.076ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
52092024-08-22T17:55:45.076ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
52102024-08-22T17:55:45.076ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
52112024-08-22T17:55:45.076ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
52122024-08-22T17:55:45.076ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
52132024-08-22T17:55:45.076ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
52142024-08-22T17:55:45.076ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
52152024-08-22T17:55:45.076ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
52162024-08-22T17:55:45.076ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
52172024-08-22T17:55:45.076ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
52182024-08-22T17:55:45.076ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
52192024-08-22T17:55:45.076ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
52202024-08-22T17:55:45.076ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
52212024-08-22T17:55:45.076ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
52222024-08-22T17:55:45.076ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
52232024-08-22T17:55:45.076ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
52242024-08-22T17:55:45.076ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
52252024-08-22T17:55:45.076ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
52262024-08-22T17:55:45.076ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
52272024-08-22T17:55:45.076ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
52282024-08-22T17:55:45.076ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
52292024-08-22T17:55:45.076ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
52302024-08-22T17:55:45.076ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
52312024-08-22T17:55:45.076ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
52322024-08-22T17:55:45.076ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
52332024-08-22T17:55:45.076ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
52342024-08-22T17:55:45.077ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 13 reason = Timeout
52352024-08-22T17:55:45.077ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
52362024-08-22T17:55:45.077ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
52372024-08-22T17:55:45.077ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("6507c79b-9a18-46c7-930d-c98e8662b87f.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("a09ea923-b85c-428d-b289-6936a0c2542c.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("c0ef020c-52c1-4b9a-a085-445fab31264f.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("c0ef020c-52c1-4b9a-a085-445fab31264f.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:3::1))) }], valid_until: Instant { tv_sec: 1933, tv_nsec: 603130271 } })
52382024-08-22T17:55:45.077ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("9488ce0a-6732-4f75-baf9-4ecfa406df91.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("afac2eab-cadf-48d0-94a7-ad562ade88c6.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("afac2eab-cadf-48d0-94a7-ad562ade88c6.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::9))) }], valid_until: Instant { tv_sec: 1933, tv_nsec: 603146921 } })
52392024-08-22T17:55:45.078ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
52402024-08-22T17:55:45.078ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
52412024-08-22T17:55:45.078ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
52422024-08-22T17:55:45.079ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
52432024-08-22T17:55:45.079ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 4.178051ms iteration = 7 status = {"errors":[],"found_port_ok":[]}
52442024-08-22T17:55:45.080ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 4.714435ms iteration = 13 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
52452024-08-22T17:55:45.080ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
52462024-08-22T17:55:45.080ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 5.258977ms iteration = 7 status = {"addresses":["[fd00:1122:3344:101::8]:5353","[fd00:1122:3344:101::9]:5353"]}
52472024-08-22T17:55:45.081ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
52482024-08-22T17:55:45.081ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
52492024-08-22T17:55:45.082ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
52502024-08-22T17:55:45.082ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
52512024-08-22T17:55:45.082ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
52522024-08-22T17:55:45.082ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } dns_group = internal
52532024-08-22T17:55:45.082ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 7.134967ms iteration = 7 status = {"addresses":["[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:3::1]:5353"]}
52542024-08-22T17:55:45.082ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
52552024-08-22T17:55:45.082ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-c961ede4-7532-4ca1-95a5-690b4d7c9b5a.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-c961ede4-7532-4ca1-95a5-690b4d7c9b5a.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1933, tv_nsec: 608250827 } })
52562024-08-22T17:55:45.082ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
52572024-08-22T17:55:45.082ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
52582024-08-22T17:55:45.083ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1050
52592024-08-22T17:55:45.084ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
52602024-08-22T17:55:45.084ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
52612024-08-22T17:55:45.087ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
52622024-08-22T17:55:45.087ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
52632024-08-22T17:55:45.087ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
52642024-08-22T17:55:45.087ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
52652024-08-22T17:55:45.087ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
52662024-08-22T17:55:45.087ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-c961ede4-7532-4ca1-95a5-690b4d7c9b5a.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-c961ede4-7532-4ca1-95a5-690b4d7c9b5a.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1933, tv_nsec: 613061522 } })
52672024-08-22T17:55:45.087ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
52682024-08-22T17:55:45.088ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1050
52692024-08-22T17:55:45.089ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
52702024-08-22T17:55:45.089ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
52712024-08-22T17:55:45.090ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
52722024-08-22T17:55:45.091ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
52732024-08-22T17:55:45.092ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
52742024-08-22T17:55:45.092ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
52752024-08-22T17:55:45.092ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
52762024-08-22T17:55:45.093ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
52772024-08-22T17:55:45.093ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
52782024-08-22T17:55:45.094ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
52792024-08-22T17:55:45.094ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2024-08-22 17:45:45.075666136 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
52802024-08-22T17:55:45.094ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 18.747016ms iteration = 7 status = {"expiration":"2024-08-22T17:45:45.075666136Z","pruned":[]}
52812024-08-22T17:55:45.094ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
52822024-08-22T17:55:45.095ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
52832024-08-22T17:55:45.096ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
52842024-08-22T17:55:45.096ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
52852024-08-22T17:55:45.096ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
52862024-08-22T17:55:45.097ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
52872024-08-22T17:55:45.097ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
52882024-08-22T17:55:45.098ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
52892024-08-22T17:55:45.099ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 21.922653ms iteration = 13 status = {"errors":[],"garbage_collect_requested":[]}
52902024-08-22T17:55:45.099ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
52912024-08-22T17:55:45.099ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
52922024-08-22T17:55:45.099ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
52932024-08-22T17:55:45.100ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
52942024-08-22T17:55:45.100ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
52952024-08-22T17:55:45.100ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
52962024-08-22T17:55:45.103ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
52972024-08-22T17:55:45.104ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
52982024-08-22T17:55:45.106ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 31.287172ms iteration = 13 status = {"region_replacement_started_err":0,"region_replacement_started_ok":0}
52992024-08-22T17:55:45.109ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
53002024-08-22T17:55:45.109ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
53012024-08-22T17:55:45.114ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
53022024-08-22T17:55:45.115ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
53032024-08-22T17:55:45.118ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 147bdc10-344c-4ee8-a219-603cfc66e6d2, lookup_type: ById(147bdc10-344c-4ee8-a219-603cfc66e6d2) } result = Ok(())
53042024-08-22T17:55:45.118ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
53052024-08-22T17:55:45.118ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
53062024-08-22T17:55:45.119ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
53072024-08-22T17:55:45.119ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
53082024-08-22T17:55:45.119ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
53092024-08-22T17:55:45.126ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
53102024-08-22T17:55:45.126ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
53112024-08-22T17:55:45.132ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
53122024-08-22T17:55:45.133ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
53132024-08-22T17:55:45.135ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
53142024-08-22T17:55:45.138ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
53152024-08-22T17:55:45.138ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0
53162024-08-22T17:55:45.138ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:189 found = 0 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0
53172024-08-22T17:55:45.138ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 63.017126ms iteration = 7 status = {"error":null,"error_count":0,"found":0,"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0}
53182024-08-22T17:55:45.138ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
53192024-08-22T17:55:45.138ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
53202024-08-22T17:55:45.139ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
53212024-08-22T17:55:45.140ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
53222024-08-22T17:55:45.140ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
53232024-08-22T17:55:45.140ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
53242024-08-22T17:55:45.141ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
53252024-08-22T17:55:45.142ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
53262024-08-22T17:55:45.143ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:55
53272024-08-22T17:55:45.143ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
53282024-08-22T17:55:45.143ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
53292024-08-22T17:55:45.143ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
53302024-08-22T17:55:45.144ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
53312024-08-22T17:55:45.144ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
53322024-08-22T17:55:45.144ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
53332024-08-22T17:55:45.144ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
53342024-08-22T17:55:45.144ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 147bdc10-344c-4ee8-a219-603cfc66e6d2 original_time_created = 2024-08-22 17:49:18.445237 UTC target_id = 147bdc10-344c-4ee8-a219-603cfc66e6d2 time_created = 2024-08-22 17:49:18.445237 UTC
53352024-08-22T17:55:45.144ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 69.412298ms iteration = 37 status = {"status":"target blueprint unchanged","target_id":"147bdc10-344c-4ee8-a219-603cfc66e6d2","time_created":"2024-08-22T17:49:18.445237Z"}
53362024-08-22T17:55:45.145ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
53372024-08-22T17:55:45.145ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
53382024-08-22T17:55:45.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
53392024-08-22T17:55:45.146ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
53402024-08-22T17:55:45.146ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
53412024-08-22T17:55:45.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
53422024-08-22T17:55:45.152ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
53432024-08-22T17:55:45.152ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
53442024-08-22T17:55:45.153ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
53452024-08-22T17:55:45.153ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
53462024-08-22T17:55:45.153ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
53472024-08-22T17:55:45.153ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
53482024-08-22T17:55:45.153ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
53492024-08-22T17:55:45.154ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
53502024-08-22T17:55:45.154ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
53512024-08-22T17:55:45.154ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
53522024-08-22T17:55:45.154ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
53532024-08-22T17:55:45.155ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
53542024-08-22T17:55:45.155ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
53552024-08-22T17:55:45.155ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:140 found = 0 not_ready_to_be_deleted = 0
53562024-08-22T17:55:45.155ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 80.404336ms iteration = 7 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
53572024-08-22T17:55:45.157ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
53582024-08-22T17:55:45.159ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
53592024-08-22T17:55:45.159ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
53602024-08-22T17:55:45.159ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
53612024-08-22T17:55:45.165ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
53622024-08-22T17:55:45.166ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
53632024-08-22T17:55:45.207ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
53642024-08-22T17:55:45.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
53652024-08-22T17:55:45.209ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1058 zone_address = fd00:1122:3344:101::2
53662024-08-22T17:55:45.209ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
53672024-08-22T17:55:45.210ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "2fabc58b-a25c-484d-b8be-675c6387d5d1", "content-length": "26", "date": "Thu, 22 Aug 2024 17:55:44 GMT"} })
53682024-08-22T17:55:45.210ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1063 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
53692024-08-22T17:55:45.211ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1095 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
53702024-08-22T17:55:45.237ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1058 zone_address = fd00:1122:3344:101::2
53712024-08-22T17:55:45.237ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
53722024-08-22T17:55:45.237ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "202c4d3b-6ddd-4cf8-9e5f-c21a7cbf3ffd", "content-length": "26", "date": "Thu, 22 Aug 2024 17:55:44 GMT"} })
53732024-08-22T17:55:45.237ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1063 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
53742024-08-22T17:55:45.239ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1095 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
53752024-08-22T17:55:45.282ZDEBGnexus (BackgroundTasks): client request SledAgent = c961ede4-7532-4ca1-95a5-690b4d7c9b5a background_task = instance_watcher body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/instances/3d8fbd34-9356-4033-8828-97c62941199a/state
53762024-08-22T17:55:45.283ZDEBGnexus (BackgroundTasks): client response SledAgent = c961ede4-7532-4ca1-95a5-690b4d7c9b5a background_task = instance_watcher result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/instances/3d8fbd34-9356-4033-8828-97c62941199a/state", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "a1130884-d286-41c6-824e-b9b02ae0dda7", "content-length": "183", "date": "Thu, 22 Aug 2024 17:55:44 GMT"} })
53772024-08-22T17:55:45.284ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 209.02446ms iteration = 13 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
53782024-08-22T17:55:45.286ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
53792024-08-22T17:55:45.286ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
53802024-08-22T17:55:45.288ZDEBGnexus (BackgroundTasks): activation complete background_task = crdb_node_id_collector elapsed = 212.86561ms iteration = 4 status = {"nsuccess":5}
53812024-08-22T17:55:45.288ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-c961ede4-7532-4ca1-95a5-690b4d7c9b5a.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-c961ede4-7532-4ca1-95a5-690b4d7c9b5a.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1933, tv_nsec: 813992890 } })
53822024-08-22T17:55:45.288ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
53832024-08-22T17:55:45.288ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
53842024-08-22T17:55:45.289ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
53852024-08-22T17:55:45.289ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
53862024-08-22T17:55:45.289ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1050 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561
53872024-08-22T17:55:45.291ZDEBGnexus (BackgroundTasks): updating instance state background_task = instance_watcher instance_id = 3d8fbd34-9356-4033-8828-97c62941199a state = Running
53882024-08-22T17:55:45.292ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
53892024-08-22T17:55:45.292ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
53902024-08-22T17:55:45.292ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
53912024-08-22T17:55:45.293ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
53922024-08-22T17:55:45.293ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
53932024-08-22T17:55:45.293ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
53942024-08-22T17:55:45.293ZINFOnexus (BackgroundTasks): received new VMM runtime state from sled agent background_task = instance_watcher file = nexus/src/app/instance.rs:1846 instance_id = 3d8fbd34-9356-4033-8828-97c62941199a migration_state = Migrations { migration_in: None, migration_out: None } propolis_id = f5632137-facd-40e2-9361-1a4348c021df vmm_state = VmmRuntimeState { state: Running, gen: Generation(3), time_updated: 2024-08-22T17:55:42.683020911Z }
53952024-08-22T17:55:45.293ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher instance_id = 3d8fbd34-9356-4033-8828-97c62941199a roles = RoleSet { roles: {} }
53962024-08-22T17:55:45.294ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher instance_id = 3d8fbd34-9356-4033-8828-97c62941199a resource = Database result = Ok(())
53972024-08-22T17:55:45.313ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 237.542188ms iteration = 37 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
53982024-08-22T17:55:45.324ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
53992024-08-22T17:55:45.324ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 7f16ce95-fbde-4896-9a22-7f61d8d01c5d dns_zone_name = oxide.test found_so_far = 0 version = 2
54002024-08-22T17:55:45.324ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
54012024-08-22T17:55:45.325ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
54022024-08-22T17:55:45.325ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:55
54032024-08-22T17:55:45.325ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:220 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
54042024-08-22T17:55:45.325ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 249.74021ms iteration = 13 status = {"destroyed_active_vmms":0,"error":null,"saga_start_failures":0,"sagas_completed":0,"sagas_failed":0,"sagas_started":0,"terminated_active_migrations":0}
54052024-08-22T17:55:45.329ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54062024-08-22T17:55:45.330ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54072024-08-22T17:55:45.332ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
54082024-08-22T17:55:45.332ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
54092024-08-22T17:55:45.332ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
54102024-08-22T17:55:45.333ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
54112024-08-22T17:55:45.333ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
54122024-08-22T17:55:45.333ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
54132024-08-22T17:55:45.336ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54142024-08-22T17:55:45.338ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
54152024-08-22T17:55:45.340ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: c961ede4-7532-4ca1-95a5-690b4d7c9b5a, lookup_type: ById(c961ede4-7532-4ca1-95a5-690b4d7c9b5a) } result = Ok(())
54162024-08-22T17:55:45.340ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
54172024-08-22T17:55:45.340ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
54182024-08-22T17:55:45.354ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
54192024-08-22T17:55:45.371ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 295.658938ms iteration = 13 status = {}
54202024-08-22T17:55:45.382ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
54212024-08-22T17:55:45.382ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "fbd0f36f-86c8-4338-a1d8-129c71f8dac1", "content-length": "1", "date": "Thu, 22 Aug 2024 17:55:44 GMT"} })
54222024-08-22T17:55:45.387ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
54232024-08-22T17:55:45.388ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
54242024-08-22T17:55:45.407ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
54252024-08-22T17:55:45.407ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
54262024-08-22T17:55:45.437ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1058 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 zone_address = fd00:1122:3344:101::2
54272024-08-22T17:55:45.437ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
54282024-08-22T17:55:45.437ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "85bfabe2-a57c-49c5-8c1e-8485b13e728e", "content-length": "26", "date": "Thu, 22 Aug 2024 17:55:44 GMT"} })
54292024-08-22T17:55:45.437ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1063 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
54302024-08-22T17:55:45.439ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1095 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561
54312024-08-22T17:55:45.447ZDEBGnexus (BackgroundTasks): client request SledAgent = c961ede4-7532-4ca1-95a5-690b4d7c9b5a background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
54322024-08-22T17:55:45.448ZDEBGnexus (BackgroundTasks): client response SledAgent = c961ede4-7532-4ca1-95a5-690b4d7c9b5a background_task = vpc_route_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/vpc-routes", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "d99835ef-08d7-4ca6-a45f-a317d0e97fe1", "content-length": "525", "date": "Thu, 22 Aug 2024 17:55:45 GMT"} })
54332024-08-22T17:55:45.448ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
54342024-08-22T17:55:45.449ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
54352024-08-22T17:55:45.452ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 376.807327ms iteration = 13 status = 0
54362024-08-22T17:55:45.452ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 377.200589ms iteration = 13 status = null
54372024-08-22T17:55:45.452ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54382024-08-22T17:55:45.453ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
54392024-08-22T17:55:45.454ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
54402024-08-22T17:55:45.456ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
54412024-08-22T17:55:45.456ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
54422024-08-22T17:55:45.456ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
54432024-08-22T17:55:45.460ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
54442024-08-22T17:55:45.460ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
54452024-08-22T17:55:45.463ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
54462024-08-22T17:55:45.463ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
54472024-08-22T17:55:45.465ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
54482024-08-22T17:55:45.466ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
54492024-08-22T17:55:45.468ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
54502024-08-22T17:55:45.469ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
54512024-08-22T17:55:45.480ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
54522024-08-22T17:55:45.480ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54532024-08-22T17:55:45.481ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
54542024-08-22T17:55:45.481ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
54552024-08-22T17:55:45.481ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = d6f65b4e-2e87-480a-af21-a69b4520828c dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
54562024-08-22T17:55:45.481ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
54572024-08-22T17:55:45.481ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
54582024-08-22T17:55:45.484ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
54592024-08-22T17:55:45.484ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
54602024-08-22T17:55:45.484ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
54612024-08-22T17:55:45.485ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
54622024-08-22T17:55:45.485ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
54632024-08-22T17:55:45.486ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 7f16ce95-fbde-4896-9a22-7f61d8d01c5d dns_zone_name = oxide.test found_so_far = 1 version = 2
54642024-08-22T17:55:45.486ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
54652024-08-22T17:55:45.486ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-08-22 17:49:58.502032 UTC dns_group = external generation = 2
54662024-08-22T17:55:45.486ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:454 pruned_instances = 0 total_completed = 1 total_failed = 0 total_incomplete = 0 total_instances = 1 update_sagas_queued = 0
54672024-08-22T17:55:45.486ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 411.137924ms iteration = 7 status = {"generation":2}
54682024-08-22T17:55:45.486ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 410.946203ms iteration = 13 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{"running":1},"pruned_instances":0,"total_instances":1,"update_sagas_queued":0}
54692024-08-22T17:55:45.486ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54702024-08-22T17:55:45.487ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
54712024-08-22T17:55:45.487ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
54722024-08-22T17:55:45.488ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
54732024-08-22T17:55:45.488ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
54742024-08-22T17:55:45.488ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
54752024-08-22T17:55:45.489ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
54762024-08-22T17:55:45.489ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
54772024-08-22T17:55:45.489ZINFOnexus (BackgroundTasks): found Silo/DNS/TLS config (changed) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-08-22T17:49:43.060798Z, time_modified: 2024-08-22T17:49:43.060798Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 10d39b39-43d3-41db-b622-3ef3e09e2dad, db_silo: Silo { identity: SiloIdentity { id: 10d39b39-43d3-41db-b622-3ef3e09e2dad, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-08-22T17:49:58.442383Z, time_modified: 2024-08-22T17:49:58.442383Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "852177ddd7117113b778290a63910551c7b25a6718e9c8fda4590027c9443fee" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 10d39b39-43d3-41db-b622-3ef3e09e2dad, db_silo: Silo { identity: SiloIdentity { id: 10d39b39-43d3-41db-b622-3ef3e09e2dad, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-08-22T17:49:58.442383Z, time_modified: 2024-08-22T17:49:58.442383Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "852177ddd7117113b778290a63910551c7b25a6718e9c8fda4590027c9443fee" }] }) } file = nexus/src/app/background/tasks/external_endpoints.rs:94
54782024-08-22T17:55:45.490ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 414.354182ms iteration = 7 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"10d39b39-43d3-41db-b622-3ef3e09e2dad","tls_certs":["852177ddd7117113b778290a63910551c7b25a6718e9c8fda4590027c9443fee"]}},"default_endpoint":{"silo_id":"10d39b39-43d3-41db-b622-3ef3e09e2dad","tls_certs":["852177ddd7117113b778290a63910551c7b25a6718e9c8fda4590027c9443fee"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
54792024-08-22T17:55:45.491ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
54802024-08-22T17:55:45.491ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
54812024-08-22T17:55:45.541ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54822024-08-22T17:55:45.544ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
54832024-08-22T17:55:45.546ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
54842024-08-22T17:55:45.546ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
54852024-08-22T17:55:45.546ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
54862024-08-22T17:55:45.546ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
54872024-08-22T17:55:45.549ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
54882024-08-22T17:55:45.549ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
54892024-08-22T17:55:45.550ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = d6f65b4e-2e87-480a-af21-a69b4520828c dns_zone_name = control-plane.oxide.internal found_so_far = 50 version = 1
54902024-08-22T17:55:45.550ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
54912024-08-22T17:55:45.550ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-08-22 17:49:57.683168 UTC dns_group = internal generation = 1
54922024-08-22T17:55:45.550ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 474.761193ms iteration = 7 status = {"generation":1}
54932024-08-22T17:55:45.551ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54942024-08-22T17:55:45.558ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = VpcRouter { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 10d39b39-43d3-41db-b622-3ef3e09e2dad, lookup_type: ById(10d39b39-43d3-41db-b622-3ef3e09e2dad) }, key: bec65549-17f3-43ff-a452-07dd35246910, lookup_type: ById(bec65549-17f3-43ff-a452-07dd35246910) }, key: 998186cd-b2c4-40e6-9d6b-ddecc09f920b, lookup_type: ById(998186cd-b2c4-40e6-9d6b-ddecc09f920b) }, key: 82138d99-0fdd-4975-9658-030c940dc2a8, lookup_type: ById(82138d99-0fdd-4975-9658-030c940dc2a8) } result = Ok(())
54952024-08-22T17:55:45.558ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
54962024-08-22T17:55:45.559ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
54972024-08-22T17:55:45.561ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
54982024-08-22T17:55:45.562ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
54992024-08-22T17:55:45.563ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
55002024-08-22T17:55:45.563ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
55012024-08-22T17:55:45.564ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
55022024-08-22T17:55:45.565ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
55032024-08-22T17:55:45.565ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::8)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "7d1729fc-f79e-4604-b9c3-3b3e505959b7", "date": "Thu, 22 Aug 2024 17:55:45 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
55042024-08-22T17:55:45.565ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
55052024-08-22T17:55:45.566ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
55062024-08-22T17:55:45.567ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::9)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "d74e6c78-2c21-401f-be04-81aec4fa019a", "date": "Thu, 22 Aug 2024 17:55:45 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
55072024-08-22T17:55:45.567ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
55082024-08-22T17:55:45.570ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
55092024-08-22T17:55:45.570ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 495.037138ms iteration = 9 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
55102024-08-22T17:55:45.572ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = VpcRouter { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 10d39b39-43d3-41db-b622-3ef3e09e2dad, lookup_type: ById(10d39b39-43d3-41db-b622-3ef3e09e2dad) }, key: bec65549-17f3-43ff-a452-07dd35246910, lookup_type: ById(bec65549-17f3-43ff-a452-07dd35246910) }, key: 998186cd-b2c4-40e6-9d6b-ddecc09f920b, lookup_type: ById(998186cd-b2c4-40e6-9d6b-ddecc09f920b) }, key: 82138d99-0fdd-4975-9658-030c940dc2a8, lookup_type: ById(82138d99-0fdd-4975-9658-030c940dc2a8) } result = Ok(())
55112024-08-22T17:55:45.572ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
55122024-08-22T17:55:45.572ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
55132024-08-22T17:55:45.588ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 512.65244ms iteration = 14 status = {"physical_disks_added":0}
55142024-08-22T17:55:45.590ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
55152024-08-22T17:55:45.594ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
55162024-08-22T17:55:45.734ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
55172024-08-22T17:55:45.735ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:2::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "7c5d846b-7a38-459f-9ed2-7fb0f6eac175", "date": "Thu, 22 Aug 2024 17:55:45 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
55182024-08-22T17:55:45.735ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
55192024-08-22T17:55:45.737ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:3::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "3f98ffb6-2dfe-4ecc-8c1e-2884216d55c4", "date": "Thu, 22 Aug 2024 17:55:45 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
55202024-08-22T17:55:45.737ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
55212024-08-22T17:55:45.739ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:1::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "0322c625-7f56-4b3a-b106-cf79645fc954", "date": "Thu, 22 Aug 2024 17:55:45 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
55222024-08-22T17:55:45.739ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
55232024-08-22T17:55:45.741ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
55242024-08-22T17:55:45.741ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 666.300123ms iteration = 9 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
55252024-08-22T17:55:45.745ZDEBGnexus (BackgroundTasks): client request SledAgent = c961ede4-7532-4ca1-95a5-690b4d7c9b5a background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
55262024-08-22T17:55:45.745ZDEBGnexus (BackgroundTasks): client response SledAgent = c961ede4-7532-4ca1-95a5-690b4d7c9b5a background_task = v2p_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/v2p", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "c18de1d8-4aa6-4a5c-a1a3-c60fa6db2199", "content-length": "811", "date": "Thu, 22 Aug 2024 17:55:45 GMT"} })
55272024-08-22T17:55:45.745ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {VirtualNetworkInterfaceHost { physical_host_ip: fd00:1122:3344:101::1, virtual_ip: 172.30.0.5, virtual_mac: MacAddr(MacAddr6([168, 64, 37, 254, 87, 16])), vni: Vni(11497890) }} sled = buskin
55282024-08-22T17:55:45.745ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = buskin
55292024-08-22T17:55:45.745ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = buskin
55302024-08-22T17:55:45.746ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
55312024-08-22T17:55:45.747ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
55322024-08-22T17:55:45.747ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 672.148124ms iteration = 13 status = {}
55332024-08-22T17:55:45.748ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
55342024-08-22T17:55:45.749ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
55352024-08-22T17:55:45.751ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
55362024-08-22T17:55:45.751ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
55372024-08-22T17:55:45.753ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:147 port = SwitchPort { id: c772d372-7448-4ce0-b8d1-efa645d18ed1, rack_id: c66d93a6-e2c3-4b19-9564-3b58bad50561, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(9ff05247-84aa-47d8-8619-ce0a8dbbaf7c) } rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 switch_location = Switch0
55382024-08-22T17:55:45.753ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
55392024-08-22T17:55:45.753ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
55402024-08-22T17:55:45.753ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
55412024-08-22T17:55:45.754ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
55422024-08-22T17:55:45.756ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
55432024-08-22T17:55:45.756ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
55442024-08-22T17:55:45.757ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
55452024-08-22T17:55:45.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
55462024-08-22T17:55:45.759ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
55472024-08-22T17:55:45.759ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
55482024-08-22T17:55:45.761ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
55492024-08-22T17:55:45.762ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
55502024-08-22T17:55:45.763ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/port/qsfp0/settings", query: Some("tag=omicron"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "6a1d16e3-f275-4947-85e4-e0df52d4bb67", "content-length": "121", "date": "Thu, 22 Aug 2024 17:55:44 GMT"} })
55512024-08-22T17:55:45.763ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1714 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G } }} } switch_port_id = "qsfp0"
55522024-08-22T17:55:45.763ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1741 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 9ff05247-84aa-47d8-8619-ce0a8dbbaf7c
55532024-08-22T17:55:45.763ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:379 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
55542024-08-22T17:55:45.763ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
55552024-08-22T17:55:45.763ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:392 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 routes = {}
55562024-08-22T17:55:45.763ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:398 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 routes = {}
55572024-08-22T17:55:45.763ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:418 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561
55582024-08-22T17:55:45.763ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
55592024-08-22T17:55:45.763ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
55602024-08-22T17:55:45.765ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
55612024-08-22T17:55:45.766ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv4", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "d75dea8d-5218-49f8-b7e8-f6143f15d5ab", "content-length": "2", "date": "Thu, 22 Aug 2024 17:55:44 GMT"} })
55622024-08-22T17:55:45.766ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
55632024-08-22T17:55:45.767ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "07e03ad7-2fd4-4ae9-87d2-02b3373c5c8d", "content-length": "52", "date": "Thu, 22 Aug 2024 17:55:44 GMT"} })
55642024-08-22T17:55:45.767ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0" }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:467 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 switch_location = Switch0
55652024-08-22T17:55:45.767ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
55662024-08-22T17:55:45.770ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 10d39b39-43d3-41db-b622-3ef3e09e2dad, lookup_type: ById(10d39b39-43d3-41db-b622-3ef3e09e2dad) }, key: bec65549-17f3-43ff-a452-07dd35246910, lookup_type: ById(bec65549-17f3-43ff-a452-07dd35246910) }, key: 998186cd-b2c4-40e6-9d6b-ddecc09f920b, lookup_type: ById(998186cd-b2c4-40e6-9d6b-ddecc09f920b) }, key: 0a5b249b-e234-4c98-ac7f-6e4a8240d62f, lookup_type: ByName("default") } result = Ok(())
55672024-08-22T17:55:45.770ZDEBGnexus (BackgroundTasks): client request SledAgent = c961ede4-7532-4ca1-95a5-690b4d7c9b5a background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
55682024-08-22T17:55:45.869ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/switch-ports", query: None, fragment: None }, status: 204, headers: {"x-request-id": "10ffdbf4-7902-4b98-9819-63e9d1d727a1", "date": "Thu, 22 Aug 2024 17:55:45 GMT"} })
55692024-08-22T17:55:45.869ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:770 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 switch = Switch0
55702024-08-22T17:55:45.869ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
55712024-08-22T17:55:45.869ZDEBGnexus (BackgroundTasks): client response SledAgent = c961ede4-7532-4ca1-95a5-690b4d7c9b5a background_task = vpc_route_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/vpc-routes", query: None, fragment: None }, status: 204, headers: {"x-request-id": "643f2ff8-26b5-4a62-a8de-e433617d948c", "date": "Thu, 22 Aug 2024 17:55:45 GMT"} })
55722024-08-22T17:55:45.871ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 796.284335ms iteration = 13 status = {}
55732024-08-22T17:55:45.874ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/network-bootstore-config", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "8ba570fd-9b8e-4ee7-9c24-a3156043e79c", "content-length": "667", "date": "Thu, 22 Aug 2024 17:55:45 GMT"} })
55742024-08-22T17:55:45.874ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
55752024-08-22T17:55:45.874ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
55762024-08-22T17:55:45.878ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
55772024-08-22T17:55:45.879ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
55782024-08-22T17:55:45.882ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
55792024-08-22T17:55:45.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
55802024-08-22T17:55:45.884ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
55812024-08-22T17:55:45.885ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
55822024-08-22T17:55:45.891ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 815.969357ms iteration = 13 status = {}
55832024-08-22T17:55:46.009ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 289 local_addr = [fd00:1122:3344:101::c]:56994 method = GET remote_addr = [fd00:1122:3344:101::d]:40670 req_id = 5186fb38-37f0-4924-800b-1e709ba67436 response_code = 200 uri = /eaa260db-2b19-4ae1-97e8-5735c9f5d955
55842024-08-22T17:55:46.012ZDEBGnexus (producer-registration-task): registering / renewing oximeter producer lease with Nexus
55852024-08-22T17:55:46.012ZDEBGnexus (producer-registration-task): using nexus address for registration addr = [fd00:1122:3344:101::c]:12221
55862024-08-22T17:55:46.044ZDEBGnexus (producer-registration-task): client request body = Some(Body) method = POST uri = http://[fd00:1122:3344:101::c]:12221/metrics/producers
55872024-08-22T17:55:46.044ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::c]:53606
55882024-08-22T17:55:46.044ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:53606 req_id = 7a18d0ce-ca2b-4814-be49-405e476b860f roles = RoleSet { roles: {} } uri = /metrics/producers
55892024-08-22T17:55:46.044ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:53606 req_id = 7a18d0ce-ca2b-4814-be49-405e476b860f resource = Database result = Ok(()) uri = /metrics/producers
55902024-08-22T17:55:46.080ZINFOnexus: registered oximeter collector client file = nexus/src/app/oximeter.rs:338 id = 7fc6f696-ffb5-46eb-ae34-da7098d2b35a
55912024-08-22T17:55:46.080ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:53606 req_id = 7a18d0ce-ca2b-4814-be49-405e476b860f roles = RoleSet { roles: {} } uri = /metrics/producers
55922024-08-22T17:55:46.080ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:53606 req_id = 7a18d0ce-ca2b-4814-be49-405e476b860f resource = Database result = Ok(()) uri = /metrics/producers
55932024-08-22T17:55:46.112ZDEBGnexus: client request body = Some(Body) method = POST oximeter-collector = 7fc6f696-ffb5-46eb-ae34-da7098d2b35a uri = http://[fd00:1122:3344:101::d]:12223/producers
55942024-08-22T17:55:46.113ZDEBGnexus: client response oximeter-collector = 7fc6f696-ffb5-46eb-ae34-da7098d2b35a result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::d)), port: Some(12223), path: "/producers", query: None, fragment: None }, status: 204, headers: {"x-request-id": "fe1d4104-0622-48ef-bea9-f24c3b0c8bcc", "date": "Thu, 22 Aug 2024 17:55:46 GMT"} })
55952024-08-22T17:55:46.113ZINFOnexus: assigned collector to new producer collector_id = 7fc6f696-ffb5-46eb-ae34-da7098d2b35a file = nexus/src/app/oximeter.rs:125 producer_id = eaa260db-2b19-4ae1-97e8-5735c9f5d955
55962024-08-22T17:55:46.115ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 71376 local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:53606 req_id = 7a18d0ce-ca2b-4814-be49-405e476b860f response_code = 201 uri = /metrics/producers
55972024-08-22T17:55:46.116ZDEBGnexus (producer-registration-task): client response result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::c)), port: Some(12221), path: "/metrics/producers", query: None, fragment: None }, status: 201, headers: {"content-type": "application/json", "x-request-id": "7a18d0ce-ca2b-4814-be49-405e476b860f", "content-length": "41", "date": "Thu, 22 Aug 2024 17:55:46 GMT"} })
55982024-08-22T17:55:46.117ZDEBGnexus (producer-registration-task): registered with nexus successfully
55992024-08-22T17:55:46.117ZDEBGnexus (producer-registration-task): pausing until time to renew lease lease_duration = 600s wait_period = 150s
56002024-08-22T17:55:55.075ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 38 reason = Timeout
56012024-08-22T17:55:55.075ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 38 reason = Timeout
56022024-08-22T17:55:55.075ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
56032024-08-22T17:55:55.075ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
56042024-08-22T17:55:55.075ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
56052024-08-22T17:55:55.076ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
56062024-08-22T17:55:55.078ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
56072024-08-22T17:55:55.080ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
56082024-08-22T17:55:55.080ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
56092024-08-22T17:55:55.080ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
56102024-08-22T17:55:55.082ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
56112024-08-22T17:55:55.082ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
56122024-08-22T17:55:55.084ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
56132024-08-22T17:55:55.086ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 147bdc10-344c-4ee8-a219-603cfc66e6d2, lookup_type: ById(147bdc10-344c-4ee8-a219-603cfc66e6d2) } result = Ok(())
56142024-08-22T17:55:55.086ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
56152024-08-22T17:55:55.086ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
56162024-08-22T17:55:55.097ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 147bdc10-344c-4ee8-a219-603cfc66e6d2 original_time_created = 2024-08-22 17:49:18.445237 UTC target_id = 147bdc10-344c-4ee8-a219-603cfc66e6d2 time_created = 2024-08-22 17:49:18.445237 UTC
56172024-08-22T17:55:55.098ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 22.384126ms iteration = 38 status = {"status":"target blueprint unchanged","target_id":"147bdc10-344c-4ee8-a219-603cfc66e6d2","time_created":"2024-08-22T17:49:18.445237Z"}
56182024-08-22T17:55:55.108ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
56192024-08-22T17:55:55.108ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
56202024-08-22T17:55:55.132ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 56.723984ms iteration = 38 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
56212024-08-22T17:55:56.115ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 274 local_addr = [fd00:1122:3344:101::c]:56994 method = GET remote_addr = [fd00:1122:3344:101::d]:40670 req_id = 79abe605-f97c-4478-82d4-2757bf6c0322 response_code = 200 uri = /eaa260db-2b19-4ae1-97e8-5735c9f5d955
56222024-08-22T17:56:05.076ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 39 reason = Timeout
56232024-08-22T17:56:05.076ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 39 reason = Timeout
56242024-08-22T17:56:05.076ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
56252024-08-22T17:56:05.076ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
56262024-08-22T17:56:05.076ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
56272024-08-22T17:56:05.076ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
56282024-08-22T17:56:05.078ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
56292024-08-22T17:56:05.080ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
56302024-08-22T17:56:05.080ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
56312024-08-22T17:56:05.081ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
56322024-08-22T17:56:05.083ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
56332024-08-22T17:56:05.083ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
56342024-08-22T17:56:05.105ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
56352024-08-22T17:56:05.105ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
56362024-08-22T17:56:05.107ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 31.237341ms iteration = 39 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
56372024-08-22T17:56:05.117ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
56382024-08-22T17:56:05.119ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 147bdc10-344c-4ee8-a219-603cfc66e6d2, lookup_type: ById(147bdc10-344c-4ee8-a219-603cfc66e6d2) } result = Ok(())
56392024-08-22T17:56:05.119ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
56402024-08-22T17:56:05.120ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
56412024-08-22T17:56:05.128ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 147bdc10-344c-4ee8-a219-603cfc66e6d2 original_time_created = 2024-08-22 17:49:18.445237 UTC target_id = 147bdc10-344c-4ee8-a219-603cfc66e6d2 time_created = 2024-08-22 17:49:18.445237 UTC
56422024-08-22T17:56:05.128ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 51.741028ms iteration = 39 status = {"status":"target blueprint unchanged","target_id":"147bdc10-344c-4ee8-a219-603cfc66e6d2","time_created":"2024-08-22T17:49:18.445237Z"}
56432024-08-22T17:56:06.116ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 440 local_addr = [fd00:1122:3344:101::c]:56994 method = GET remote_addr = [fd00:1122:3344:101::d]:40670 req_id = ce9c3542-3920-4f63-8260-35028b13732f response_code = 200 uri = /eaa260db-2b19-4ae1-97e8-5735c9f5d955
56442024-08-22T17:56:15.075ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 40 reason = Timeout
56452024-08-22T17:56:15.075ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 14 reason = Timeout
56462024-08-22T17:56:15.075ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
56472024-08-22T17:56:15.075ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
56482024-08-22T17:56:15.076ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 14 reason = Timeout
56492024-08-22T17:56:15.076ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 15 reason = Timeout
56502024-08-22T17:56:15.076ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
56512024-08-22T17:56:15.076ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
56522024-08-22T17:56:15.076ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 14 reason = Timeout
56532024-08-22T17:56:15.076ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 14 reason = Timeout
56542024-08-22T17:56:15.076ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
56552024-08-22T17:56:15.076ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 14 reason = Timeout
56562024-08-22T17:56:15.076ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
56572024-08-22T17:56:15.076ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 14 reason = Timeout
56582024-08-22T17:56:15.076ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 14 reason = Timeout
56592024-08-22T17:56:15.076ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
56602024-08-22T17:56:15.076ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
56612024-08-22T17:56:15.076ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 14 reason = Timeout
56622024-08-22T17:56:15.076ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
56632024-08-22T17:56:15.076ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 14 reason = Timeout
56642024-08-22T17:56:15.076ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
56652024-08-22T17:56:15.076ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 14 reason = Timeout
56662024-08-22T17:56:15.076ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 40 reason = Timeout
56672024-08-22T17:56:15.076ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
56682024-08-22T17:56:15.076ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
56692024-08-22T17:56:15.076ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 14 reason = Timeout
56702024-08-22T17:56:15.076ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
56712024-08-22T17:56:15.076ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
56722024-08-22T17:56:15.076ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
56732024-08-22T17:56:15.076ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
56742024-08-22T17:56:15.076ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
56752024-08-22T17:56:15.076ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
56762024-08-22T17:56:15.076ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
56772024-08-22T17:56:15.076ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
56782024-08-22T17:56:15.076ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
56792024-08-22T17:56:15.076ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
56802024-08-22T17:56:15.076ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
56812024-08-22T17:56:15.076ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
56822024-08-22T17:56:15.076ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
56832024-08-22T17:56:15.076ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
56842024-08-22T17:56:15.076ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
56852024-08-22T17:56:15.078ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
56862024-08-22T17:56:15.078ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 14 reason = Timeout
56872024-08-22T17:56:15.078ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
56882024-08-22T17:56:15.078ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
56892024-08-22T17:56:15.079ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
56902024-08-22T17:56:15.079ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
56912024-08-22T17:56:15.079ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.831739ms iteration = 14 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
56922024-08-22T17:56:15.080ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-c961ede4-7532-4ca1-95a5-690b4d7c9b5a.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-c961ede4-7532-4ca1-95a5-690b4d7c9b5a.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1963, tv_nsec: 605891359 } })
56932024-08-22T17:56:15.081ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1050
56942024-08-22T17:56:15.082ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
56952024-08-22T17:56:15.082ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
56962024-08-22T17:56:15.082ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
56972024-08-22T17:56:15.082ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
56982024-08-22T17:56:15.082ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
56992024-08-22T17:56:15.082ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
57002024-08-22T17:56:15.083ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
57012024-08-22T17:56:15.083ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
57022024-08-22T17:56:15.083ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
57032024-08-22T17:56:15.083ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
57042024-08-22T17:56:15.089ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-c961ede4-7532-4ca1-95a5-690b4d7c9b5a.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-c961ede4-7532-4ca1-95a5-690b4d7c9b5a.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1963, tv_nsec: 614007451 } })
57052024-08-22T17:56:15.089ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
57062024-08-22T17:56:15.089ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
57072024-08-22T17:56:15.089ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
57082024-08-22T17:56:15.090ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1050
57092024-08-22T17:56:15.091ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
57102024-08-22T17:56:15.093ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
57112024-08-22T17:56:15.093ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
57122024-08-22T17:56:15.093ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
57132024-08-22T17:56:15.093ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:55
57142024-08-22T17:56:15.093ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
57152024-08-22T17:56:15.093ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
57162024-08-22T17:56:15.094ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
57172024-08-22T17:56:15.101ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
57182024-08-22T17:56:15.102ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
57192024-08-22T17:56:15.104ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
57202024-08-22T17:56:15.108ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 147bdc10-344c-4ee8-a219-603cfc66e6d2, lookup_type: ById(147bdc10-344c-4ee8-a219-603cfc66e6d2) } result = Ok(())
57212024-08-22T17:56:15.108ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
57222024-08-22T17:56:15.108ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
57232024-08-22T17:56:15.109ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
57242024-08-22T17:56:15.110ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
57252024-08-22T17:56:15.110ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
57262024-08-22T17:56:15.110ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
57272024-08-22T17:56:15.111ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-c961ede4-7532-4ca1-95a5-690b4d7c9b5a.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-c961ede4-7532-4ca1-95a5-690b4d7c9b5a.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1963, tv_nsec: 636315866 } })
57282024-08-22T17:56:15.112ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
57292024-08-22T17:56:15.112ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1050 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561
57302024-08-22T17:56:15.115ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
57312024-08-22T17:56:15.115ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
57322024-08-22T17:56:15.115ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
57332024-08-22T17:56:15.124ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 147bdc10-344c-4ee8-a219-603cfc66e6d2 original_time_created = 2024-08-22 17:49:18.445237 UTC target_id = 147bdc10-344c-4ee8-a219-603cfc66e6d2 time_created = 2024-08-22 17:49:18.445237 UTC
57342024-08-22T17:56:15.124ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 49.060234ms iteration = 40 status = {"status":"target blueprint unchanged","target_id":"147bdc10-344c-4ee8-a219-603cfc66e6d2","time_created":"2024-08-22T17:49:18.445237Z"}
57352024-08-22T17:56:15.126ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
57362024-08-22T17:56:15.126ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
57372024-08-22T17:56:15.136ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
57382024-08-22T17:56:15.136ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
57392024-08-22T17:56:15.136ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
57402024-08-22T17:56:15.137ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
57412024-08-22T17:56:15.139ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
57422024-08-22T17:56:15.139ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
57432024-08-22T17:56:15.139ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
57442024-08-22T17:56:15.141ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
57452024-08-22T17:56:15.141ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
57462024-08-22T17:56:15.141ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
57472024-08-22T17:56:15.141ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
57482024-08-22T17:56:15.142ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
57492024-08-22T17:56:15.142ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
57502024-08-22T17:56:15.144ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 68.062432ms iteration = 14 status = {"region_replacement_started_err":0,"region_replacement_started_ok":0}
57512024-08-22T17:56:15.144ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:55
57522024-08-22T17:56:15.144ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:220 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
57532024-08-22T17:56:15.144ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 68.694595ms iteration = 14 status = {"destroyed_active_vmms":0,"error":null,"saga_start_failures":0,"sagas_completed":0,"sagas_failed":0,"sagas_started":0,"terminated_active_migrations":0}
57542024-08-22T17:56:15.145ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
57552024-08-22T17:56:15.145ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
57562024-08-22T17:56:15.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
57572024-08-22T17:56:15.158ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 79.922713ms iteration = 14 status = {"errors":[],"garbage_collect_requested":[]}
57582024-08-22T17:56:15.167ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 90.99011ms iteration = 40 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
57592024-08-22T17:56:15.174ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1058 zone_address = fd00:1122:3344:101::2
57602024-08-22T17:56:15.174ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
57612024-08-22T17:56:15.175ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "e4f22fe5-010b-44d4-8fe1-ad69382ceb50", "content-length": "26", "date": "Thu, 22 Aug 2024 17:56:14 GMT"} })
57622024-08-22T17:56:15.175ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1063 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
57632024-08-22T17:56:15.176ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1095 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
57642024-08-22T17:56:15.181ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 105.082063ms iteration = 14 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
57652024-08-22T17:56:15.183ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
57662024-08-22T17:56:15.185ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: c961ede4-7532-4ca1-95a5-690b4d7c9b5a, lookup_type: ById(c961ede4-7532-4ca1-95a5-690b4d7c9b5a) } result = Ok(())
57672024-08-22T17:56:15.185ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
57682024-08-22T17:56:15.186ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
57692024-08-22T17:56:15.189ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:454 pruned_instances = 1 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
57702024-08-22T17:56:15.189ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 113.157925ms iteration = 14 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":1,"total_instances":0,"update_sagas_queued":0}
57712024-08-22T17:56:15.202ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
57722024-08-22T17:56:15.202ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
57732024-08-22T17:56:15.223ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 147.999215ms iteration = 15 status = {"physical_disks_added":0}
57742024-08-22T17:56:15.238ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1058 zone_address = fd00:1122:3344:101::2
57752024-08-22T17:56:15.238ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
57762024-08-22T17:56:15.239ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "d12adec4-4519-4dd4-82e7-85d069c34649", "content-length": "26", "date": "Thu, 22 Aug 2024 17:56:14 GMT"} })
57772024-08-22T17:56:15.239ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1063 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
57782024-08-22T17:56:15.240ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1095 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
57792024-08-22T17:56:15.287ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1058 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 zone_address = fd00:1122:3344:101::2
57802024-08-22T17:56:15.287ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
57812024-08-22T17:56:15.288ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "caa0e0aa-4685-4a22-8255-d4da8939bb4a", "content-length": "26", "date": "Thu, 22 Aug 2024 17:56:14 GMT"} })
57822024-08-22T17:56:15.288ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1063 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
57832024-08-22T17:56:15.290ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1095 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561
57842024-08-22T17:56:15.308ZDEBGnexus (BackgroundTasks): client request SledAgent = c961ede4-7532-4ca1-95a5-690b4d7c9b5a background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
57852024-08-22T17:56:15.309ZDEBGnexus (BackgroundTasks): client response SledAgent = c961ede4-7532-4ca1-95a5-690b4d7c9b5a background_task = vpc_route_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/vpc-routes", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "306fe6fb-01ef-4570-a315-79a755f5cf07", "content-length": "372", "date": "Thu, 22 Aug 2024 17:56:14 GMT"} })
57862024-08-22T17:56:15.309ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
57872024-08-22T17:56:15.310ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
57882024-08-22T17:56:15.310ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 234.781323ms iteration = 14 status = 0
57892024-08-22T17:56:15.312ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
57902024-08-22T17:56:15.312ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
57912024-08-22T17:56:15.314ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
57922024-08-22T17:56:15.314ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
57932024-08-22T17:56:15.318ZDEBGnexus (BackgroundTasks): client request SledAgent = c961ede4-7532-4ca1-95a5-690b4d7c9b5a background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
57942024-08-22T17:56:15.318ZDEBGnexus (BackgroundTasks): client response SledAgent = c961ede4-7532-4ca1-95a5-690b4d7c9b5a background_task = vpc_route_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/vpc-routes", query: None, fragment: None }, status: 204, headers: {"x-request-id": "bba8f3c9-1dd0-4a4b-b4ef-d2de72230838", "date": "Thu, 22 Aug 2024 17:56:14 GMT"} })
57952024-08-22T17:56:15.319ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 243.407638ms iteration = 14 status = {}
57962024-08-22T17:56:15.332ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 256.443485ms iteration = 14 status = {}
57972024-08-22T17:56:15.340ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
57982024-08-22T17:56:15.340ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "ac58f25b-1048-4279-9962-65b35c8270a9", "content-length": "2", "date": "Thu, 22 Aug 2024 17:56:15 GMT"} })
57992024-08-22T17:56:15.343ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
58002024-08-22T17:56:15.343ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
58012024-08-22T17:56:15.343ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
58022024-08-22T17:56:15.343ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
58032024-08-22T17:56:15.345ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 269.408492ms iteration = 14 status = null
58042024-08-22T17:56:15.368ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
58052024-08-22T17:56:15.369ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
58062024-08-22T17:56:15.369ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
58072024-08-22T17:56:15.369ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
58082024-08-22T17:56:15.393ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
58092024-08-22T17:56:15.394ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
58102024-08-22T17:56:15.394ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
58112024-08-22T17:56:15.395ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
58122024-08-22T17:56:15.416ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
58132024-08-22T17:56:15.416ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
58142024-08-22T17:56:15.418ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:147 port = SwitchPort { id: c772d372-7448-4ce0-b8d1-efa645d18ed1, rack_id: c66d93a6-e2c3-4b19-9564-3b58bad50561, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(9ff05247-84aa-47d8-8619-ce0a8dbbaf7c) } rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 switch_location = Switch0
58152024-08-22T17:56:15.418ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
58162024-08-22T17:56:15.419ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
58172024-08-22T17:56:15.427ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
58182024-08-22T17:56:15.428ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/port/qsfp0/settings", query: Some("tag=omicron"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "4e229740-dd5b-4cd8-be7e-6a98c0671567", "content-length": "121", "date": "Thu, 22 Aug 2024 17:56:15 GMT"} })
58192024-08-22T17:56:15.428ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1714 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G } }} } switch_port_id = "qsfp0"
58202024-08-22T17:56:15.428ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1741 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 9ff05247-84aa-47d8-8619-ce0a8dbbaf7c
58212024-08-22T17:56:15.429ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:379 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
58222024-08-22T17:56:15.429ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
58232024-08-22T17:56:15.429ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:392 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 routes = {}
58242024-08-22T17:56:15.429ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:398 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 routes = {}
58252024-08-22T17:56:15.429ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:418 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561
58262024-08-22T17:56:15.429ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
58272024-08-22T17:56:15.429ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
58282024-08-22T17:56:15.431ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
58292024-08-22T17:56:15.431ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv4", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "164d17c2-93da-45d2-a24a-807be3e5eaa0", "content-length": "2", "date": "Thu, 22 Aug 2024 17:56:15 GMT"} })
58302024-08-22T17:56:15.431ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
58312024-08-22T17:56:15.431ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "63e875f6-c807-40c2-b56d-4c2f19a1d01f", "content-length": "52", "date": "Thu, 22 Aug 2024 17:56:15 GMT"} })
58322024-08-22T17:56:15.431ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0" }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:467 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 switch_location = Switch0
58332024-08-22T17:56:15.431ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
58342024-08-22T17:56:15.475ZDEBGnexus (BackgroundTasks): client request SledAgent = c961ede4-7532-4ca1-95a5-690b4d7c9b5a background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
58352024-08-22T17:56:15.532ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/switch-ports", query: None, fragment: None }, status: 204, headers: {"x-request-id": "5a0a16d2-cb27-4169-bf64-fef4350f80b5", "date": "Thu, 22 Aug 2024 17:56:14 GMT"} })
58362024-08-22T17:56:15.532ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:770 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 switch = Switch0
58372024-08-22T17:56:15.532ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
58382024-08-22T17:56:15.533ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/network-bootstore-config", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "f97407f3-6ea7-4f42-908f-2c4e324cefdb", "content-length": "667", "date": "Thu, 22 Aug 2024 17:56:15 GMT"} })
58392024-08-22T17:56:15.533ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
58402024-08-22T17:56:15.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
58412024-08-22T17:56:15.534ZDEBGnexus (BackgroundTasks): client response SledAgent = c961ede4-7532-4ca1-95a5-690b4d7c9b5a background_task = v2p_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/v2p", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "d4407ce0-91b2-445a-97ab-c437b23ae21e", "content-length": "691", "date": "Thu, 22 Aug 2024 17:56:15 GMT"} })
58422024-08-22T17:56:15.534ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = buskin
58432024-08-22T17:56:15.534ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = buskin
58442024-08-22T17:56:15.534ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = buskin
58452024-08-22T17:56:15.536ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 460.230398ms iteration = 14 status = {}
58462024-08-22T17:56:15.536ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
58472024-08-22T17:56:15.537ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
58482024-08-22T17:56:15.539ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
58492024-08-22T17:56:15.540ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
58502024-08-22T17:56:15.541ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
58512024-08-22T17:56:15.542ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
58522024-08-22T17:56:15.551ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 475.580287ms iteration = 14 status = {}
58532024-08-22T17:56:16.116ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 667 local_addr = [fd00:1122:3344:101::c]:56994 method = GET remote_addr = [fd00:1122:3344:101::d]:40670 req_id = e8e6906e-3882-428b-8a9d-f822311e9593 response_code = 200 uri = /eaa260db-2b19-4ae1-97e8-5735c9f5d955
58542024-08-22T17:56:25.076ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 41 reason = Timeout
58552024-08-22T17:56:25.076ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
58562024-08-22T17:56:25.076ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 41 reason = Timeout
58572024-08-22T17:56:25.076ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
58582024-08-22T17:56:25.077ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
58592024-08-22T17:56:25.077ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
58602024-08-22T17:56:25.079ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
58612024-08-22T17:56:25.082ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
58622024-08-22T17:56:25.082ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
58632024-08-22T17:56:25.082ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
58642024-08-22T17:56:25.084ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
58652024-08-22T17:56:25.084ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
58662024-08-22T17:56:25.086ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
58672024-08-22T17:56:25.088ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 147bdc10-344c-4ee8-a219-603cfc66e6d2, lookup_type: ById(147bdc10-344c-4ee8-a219-603cfc66e6d2) } result = Ok(())
58682024-08-22T17:56:25.088ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
58692024-08-22T17:56:25.088ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
58702024-08-22T17:56:25.094ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 147bdc10-344c-4ee8-a219-603cfc66e6d2 original_time_created = 2024-08-22 17:49:18.445237 UTC target_id = 147bdc10-344c-4ee8-a219-603cfc66e6d2 time_created = 2024-08-22 17:49:18.445237 UTC
58712024-08-22T17:56:25.094ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 18.052923ms iteration = 41 status = {"status":"target blueprint unchanged","target_id":"147bdc10-344c-4ee8-a219-603cfc66e6d2","time_created":"2024-08-22T17:49:18.445237Z"}
58722024-08-22T17:56:25.103ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
58732024-08-22T17:56:25.103ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
58742024-08-22T17:56:25.128ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 52.27027ms iteration = 41 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
58752024-08-22T17:56:26.116ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 339 local_addr = [fd00:1122:3344:101::c]:56994 method = GET remote_addr = [fd00:1122:3344:101::d]:40670 req_id = 2b38041c-2c20-4c26-8972-b51cf2cea125 response_code = 200 uri = /eaa260db-2b19-4ae1-97e8-5735c9f5d955
58762024-08-22T17:56:35.077ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 42 reason = Timeout
58772024-08-22T17:56:35.077ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 42 reason = Timeout
58782024-08-22T17:56:35.077ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
58792024-08-22T17:56:35.077ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
58802024-08-22T17:56:35.077ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
58812024-08-22T17:56:35.077ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
58822024-08-22T17:56:35.079ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
58832024-08-22T17:56:35.081ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
58842024-08-22T17:56:35.081ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
58852024-08-22T17:56:35.081ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
58862024-08-22T17:56:35.083ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
58872024-08-22T17:56:35.084ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
58882024-08-22T17:56:35.085ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
58892024-08-22T17:56:35.087ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 147bdc10-344c-4ee8-a219-603cfc66e6d2, lookup_type: ById(147bdc10-344c-4ee8-a219-603cfc66e6d2) } result = Ok(())
58902024-08-22T17:56:35.087ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
58912024-08-22T17:56:35.087ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
58922024-08-22T17:56:35.098ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 147bdc10-344c-4ee8-a219-603cfc66e6d2 original_time_created = 2024-08-22 17:49:18.445237 UTC target_id = 147bdc10-344c-4ee8-a219-603cfc66e6d2 time_created = 2024-08-22 17:49:18.445237 UTC
58932024-08-22T17:56:35.098ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 21.198979ms iteration = 42 status = {"status":"target blueprint unchanged","target_id":"147bdc10-344c-4ee8-a219-603cfc66e6d2","time_created":"2024-08-22T17:49:18.445237Z"}
58942024-08-22T17:56:35.107ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
58952024-08-22T17:56:35.107ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
58962024-08-22T17:56:35.138ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 60.995885ms iteration = 42 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
58972024-08-22T17:56:36.116ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 222 local_addr = [fd00:1122:3344:101::c]:56994 method = GET remote_addr = [fd00:1122:3344:101::d]:40670 req_id = c614b96b-11dd-4048-93b0-0be11f30555e response_code = 200 uri = /eaa260db-2b19-4ae1-97e8-5735c9f5d955
58982024-08-22T17:56:45.077ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 8 reason = Timeout
58992024-08-22T17:56:45.077ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 15 reason = Timeout
59002024-08-22T17:56:45.077ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
59012024-08-22T17:56:45.077ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 8 reason = Timeout
59022024-08-22T17:56:45.077ZINFOnexus (BackgroundTasks): Abandoned VMM reaper running background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:76
59032024-08-22T17:56:45.077ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 15 reason = Timeout
59042024-08-22T17:56:45.077ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
59052024-08-22T17:56:45.077ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
59062024-08-22T17:56:45.077ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 43 reason = Timeout
59072024-08-22T17:56:45.077ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 8 reason = Timeout
59082024-08-22T17:56:45.077ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
59092024-08-22T17:56:45.077ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
59102024-08-22T17:56:45.077ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 10 reason = Timeout
59112024-08-22T17:56:45.077ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 8 reason = Timeout
59122024-08-22T17:56:45.077ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:56
59132024-08-22T17:56:45.077ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 43 reason = Timeout
59142024-08-22T17:56:45.077ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
59152024-08-22T17:56:45.077ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 9 reason = Timeout
59162024-08-22T17:56:45.077ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
59172024-08-22T17:56:45.077ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:80 reason = blueprint disabled target_id = 147bdc10-344c-4ee8-a219-603cfc66e6d2
59182024-08-22T17:56:45.077ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 13.04\u{b5}s iteration = 9 status = {"error":"blueprint disabled","target_id":"147bdc10-344c-4ee8-a219-603cfc66e6d2"}
59192024-08-22T17:56:45.077ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 8 reason = Timeout
59202024-08-22T17:56:45.077ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 8 reason = Timeout
59212024-08-22T17:56:45.077ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
59222024-08-22T17:56:45.077ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 15 reason = Timeout
59232024-08-22T17:56:45.077ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 15 reason = Timeout
59242024-08-22T17:56:45.077ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
59252024-08-22T17:56:45.077ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
59262024-08-22T17:56:45.077ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
59272024-08-22T17:56:45.077ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 10 reason = Timeout
59282024-08-22T17:56:45.077ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 15 reason = Timeout
59292024-08-22T17:56:45.077ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
59302024-08-22T17:56:45.077ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 8 reason = Timeout
59312024-08-22T17:56:45.077ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
59322024-08-22T17:56:45.077ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
59332024-08-22T17:56:45.077ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 15 reason = Timeout
59342024-08-22T17:56:45.077ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
59352024-08-22T17:56:45.077ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
59362024-08-22T17:56:45.077ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 15 reason = Timeout
59372024-08-22T17:56:45.077ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
59382024-08-22T17:56:45.077ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 15 reason = Timeout
59392024-08-22T17:56:45.077ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
59402024-08-22T17:56:45.077ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 16 reason = Timeout
59412024-08-22T17:56:45.077ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 8 reason = Timeout
59422024-08-22T17:56:45.077ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
59432024-08-22T17:56:45.077ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
59442024-08-22T17:56:45.077ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 15 reason = Timeout
59452024-08-22T17:56:45.077ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
59462024-08-22T17:56:45.077ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 15 reason = Timeout
59472024-08-22T17:56:45.077ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 8 reason = Timeout
59482024-08-22T17:56:45.077ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
59492024-08-22T17:56:45.077ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2024-08-22 17:46:45.077520185 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
59502024-08-22T17:56:45.077ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 15 reason = Timeout
59512024-08-22T17:56:45.077ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
59522024-08-22T17:56:45.077ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
59532024-08-22T17:56:45.077ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
59542024-08-22T17:56:45.077ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
59552024-08-22T17:56:45.077ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
59562024-08-22T17:56:45.077ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
59572024-08-22T17:56:45.077ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
59582024-08-22T17:56:45.077ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
59592024-08-22T17:56:45.077ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
59602024-08-22T17:56:45.077ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
59612024-08-22T17:56:45.077ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
59622024-08-22T17:56:45.077ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
59632024-08-22T17:56:45.078ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
59642024-08-22T17:56:45.078ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
59652024-08-22T17:56:45.078ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
59662024-08-22T17:56:45.078ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
59672024-08-22T17:56:45.078ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
59682024-08-22T17:56:45.078ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
59692024-08-22T17:56:45.078ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
59702024-08-22T17:56:45.078ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
59712024-08-22T17:56:45.078ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
59722024-08-22T17:56:45.078ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 15 reason = Timeout
59732024-08-22T17:56:45.078ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
59742024-08-22T17:56:45.078ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
59752024-08-22T17:56:45.079ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("6507c79b-9a18-46c7-930d-c98e8662b87f.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("a09ea923-b85c-428d-b289-6936a0c2542c.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("c0ef020c-52c1-4b9a-a085-445fab31264f.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("c0ef020c-52c1-4b9a-a085-445fab31264f.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:3::1))) }], valid_until: Instant { tv_sec: 1993, tv_nsec: 603308209 } })
59762024-08-22T17:56:45.079ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("9488ce0a-6732-4f75-baf9-4ecfa406df91.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("afac2eab-cadf-48d0-94a7-ad562ade88c6.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("afac2eab-cadf-48d0-94a7-ad562ade88c6.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::9))) }], valid_until: Instant { tv_sec: 1993, tv_nsec: 603319549 } })
59772024-08-22T17:56:45.080ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
59782024-08-22T17:56:45.081ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.714889ms iteration = 15 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
59792024-08-22T17:56:45.081ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 4.556483ms iteration = 8 status = {"errors":[],"found_port_ok":[]}
59802024-08-22T17:56:45.081ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
59812024-08-22T17:56:45.082ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
59822024-08-22T17:56:45.082ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
59832024-08-22T17:56:45.082ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
59842024-08-22T17:56:45.082ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
59852024-08-22T17:56:45.083ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
59862024-08-22T17:56:45.084ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
59872024-08-22T17:56:45.084ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
59882024-08-22T17:56:45.084ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 7.060106ms iteration = 8 status = {"addresses":["[fd00:1122:3344:101::8]:5353","[fd00:1122:3344:101::9]:5353"]}
59892024-08-22T17:56:45.084ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
59902024-08-22T17:56:45.084ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
59912024-08-22T17:56:45.085ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
59922024-08-22T17:56:45.087ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
59932024-08-22T17:56:45.087ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
59942024-08-22T17:56:45.087ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
59952024-08-22T17:56:45.087ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } dns_group = internal
59962024-08-22T17:56:45.087ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 10.231323ms iteration = 8 status = {"addresses":["[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:3::1]:5353"]}
59972024-08-22T17:56:45.087ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-c961ede4-7532-4ca1-95a5-690b4d7c9b5a.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-c961ede4-7532-4ca1-95a5-690b4d7c9b5a.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1993, tv_nsec: 611651762 } })
59982024-08-22T17:56:45.090ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1050
59992024-08-22T17:56:45.094ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
60002024-08-22T17:56:45.097ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
60012024-08-22T17:56:45.097ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
60022024-08-22T17:56:45.097ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
60032024-08-22T17:56:45.099ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-c961ede4-7532-4ca1-95a5-690b4d7c9b5a.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-c961ede4-7532-4ca1-95a5-690b4d7c9b5a.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1993, tv_nsec: 623677094 } })
60042024-08-22T17:56:45.101ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1050
60052024-08-22T17:56:45.103ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2024-08-22 17:46:45.077520185 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
60062024-08-22T17:56:45.103ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 25.753913ms iteration = 8 status = {"expiration":"2024-08-22T17:46:45.077520185Z","pruned":[]}
60072024-08-22T17:56:45.104ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
60082024-08-22T17:56:45.107ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
60092024-08-22T17:56:45.108ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
60102024-08-22T17:56:45.110ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
60112024-08-22T17:56:45.111ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
60122024-08-22T17:56:45.112ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
60132024-08-22T17:56:45.113ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
60142024-08-22T17:56:45.114ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
60152024-08-22T17:56:45.116ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
60162024-08-22T17:56:45.116ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
60172024-08-22T17:56:45.116ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
60182024-08-22T17:56:45.118ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 147bdc10-344c-4ee8-a219-603cfc66e6d2, lookup_type: ById(147bdc10-344c-4ee8-a219-603cfc66e6d2) } result = Ok(())
60192024-08-22T17:56:45.118ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
60202024-08-22T17:56:45.119ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
60212024-08-22T17:56:45.130ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
60222024-08-22T17:56:45.133ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
60232024-08-22T17:56:45.133ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
60242024-08-22T17:56:45.133ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
60252024-08-22T17:56:45.136ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
60262024-08-22T17:56:45.136ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
60272024-08-22T17:56:45.136ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
60282024-08-22T17:56:45.138ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0
60292024-08-22T17:56:45.138ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:189 found = 0 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0
60302024-08-22T17:56:45.138ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 61.718019ms iteration = 8 status = {"error":null,"error_count":0,"found":0,"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0}
60312024-08-22T17:56:45.138ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
60322024-08-22T17:56:45.139ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
60332024-08-22T17:56:45.139ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
60342024-08-22T17:56:45.147ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:55
60352024-08-22T17:56:45.147ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
60362024-08-22T17:56:45.147ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
60372024-08-22T17:56:45.147ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
60382024-08-22T17:56:45.147ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
60392024-08-22T17:56:45.150ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
60402024-08-22T17:56:45.150ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
60412024-08-22T17:56:45.150ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
60422024-08-22T17:56:45.150ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
60432024-08-22T17:56:45.151ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 72.421854ms iteration = 15 status = {"errors":[],"garbage_collect_requested":[]}
60442024-08-22T17:56:45.151ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-c961ede4-7532-4ca1-95a5-690b4d7c9b5a.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-c961ede4-7532-4ca1-95a5-690b4d7c9b5a.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1993, tv_nsec: 675711673 } })
60452024-08-22T17:56:45.151ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 147bdc10-344c-4ee8-a219-603cfc66e6d2 original_time_created = 2024-08-22 17:49:18.445237 UTC target_id = 147bdc10-344c-4ee8-a219-603cfc66e6d2 time_created = 2024-08-22 17:49:18.445237 UTC
60462024-08-22T17:56:45.151ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 74.219563ms iteration = 43 status = {"status":"target blueprint unchanged","target_id":"147bdc10-344c-4ee8-a219-603cfc66e6d2","time_created":"2024-08-22T17:49:18.445237Z"}
60472024-08-22T17:56:45.152ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
60482024-08-22T17:56:45.152ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
60492024-08-22T17:56:45.152ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
60502024-08-22T17:56:45.153ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1050 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561
60512024-08-22T17:56:45.154ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
60522024-08-22T17:56:45.154ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
60532024-08-22T17:56:45.155ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 77.791392ms iteration = 43 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
60542024-08-22T17:56:45.155ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
60552024-08-22T17:56:45.155ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
60562024-08-22T17:56:45.157ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 79.725432ms iteration = 15 status = {"region_replacement_started_err":0,"region_replacement_started_ok":0}
60572024-08-22T17:56:45.157ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
60582024-08-22T17:56:45.157ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
60592024-08-22T17:56:45.157ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
60602024-08-22T17:56:45.157ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
60612024-08-22T17:56:45.157ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
60622024-08-22T17:56:45.158ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
60632024-08-22T17:56:45.162ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
60642024-08-22T17:56:45.162ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
60652024-08-22T17:56:45.162ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
60662024-08-22T17:56:45.164ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
60672024-08-22T17:56:45.165ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
60682024-08-22T17:56:45.165ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
60692024-08-22T17:56:45.165ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
60702024-08-22T17:56:45.165ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
60712024-08-22T17:56:45.165ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
60722024-08-22T17:56:45.166ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
60732024-08-22T17:56:45.166ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
60742024-08-22T17:56:45.166ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
60752024-08-22T17:56:45.167ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
60762024-08-22T17:56:45.168ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
60772024-08-22T17:56:45.171ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
60782024-08-22T17:56:45.171ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
60792024-08-22T17:56:45.171ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
60802024-08-22T17:56:45.172ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
60812024-08-22T17:56:45.172ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
60822024-08-22T17:56:45.172ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
60832024-08-22T17:56:45.175ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
60842024-08-22T17:56:45.176ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
60852024-08-22T17:56:45.178ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
60862024-08-22T17:56:45.178ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
60872024-08-22T17:56:45.180ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
60882024-08-22T17:56:45.184ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: c961ede4-7532-4ca1-95a5-690b4d7c9b5a, lookup_type: ById(c961ede4-7532-4ca1-95a5-690b4d7c9b5a) } result = Ok(())
60892024-08-22T17:56:45.184ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
60902024-08-22T17:56:45.185ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
60912024-08-22T17:56:45.189ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 112.409571ms iteration = 15 status = 0
60922024-08-22T17:56:45.198ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 121.249467ms iteration = 15 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
60932024-08-22T17:56:45.202ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:454 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
60942024-08-22T17:56:45.202ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 125.043297ms iteration = 15 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
60952024-08-22T17:56:45.204ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
60962024-08-22T17:56:45.205ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:140 found = 0 not_ready_to_be_deleted = 0
60972024-08-22T17:56:45.205ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 127.70815ms iteration = 8 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
60982024-08-22T17:56:45.213ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:55
60992024-08-22T17:56:45.213ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:220 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
61002024-08-22T17:56:45.213ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 135.702872ms iteration = 15 status = {"destroyed_active_vmms":0,"error":null,"saga_start_failures":0,"sagas_completed":0,"sagas_failed":0,"sagas_started":0,"terminated_active_migrations":0}
61012024-08-22T17:56:45.215ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
61022024-08-22T17:56:45.215ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
61032024-08-22T17:56:45.217ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
61042024-08-22T17:56:45.217ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = d6f65b4e-2e87-480a-af21-a69b4520828c dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
61052024-08-22T17:56:45.217ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
61062024-08-22T17:56:45.217ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
61072024-08-22T17:56:45.217ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
61082024-08-22T17:56:45.217ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
61092024-08-22T17:56:45.217ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
61102024-08-22T17:56:45.218ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
61112024-08-22T17:56:45.218ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
61122024-08-22T17:56:45.219ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
61132024-08-22T17:56:45.219ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
61142024-08-22T17:56:45.219ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
61152024-08-22T17:56:45.219ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
61162024-08-22T17:56:45.219ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1058 zone_address = fd00:1122:3344:101::2
61172024-08-22T17:56:45.219ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
61182024-08-22T17:56:45.220ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "c5cdc634-349f-453f-a82c-7f4be3d0f2b6", "content-length": "26", "date": "Thu, 22 Aug 2024 17:56:44 GMT"} })
61192024-08-22T17:56:45.220ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1063 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
61202024-08-22T17:56:45.220ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
61212024-08-22T17:56:45.221ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1095 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
61222024-08-22T17:56:45.222ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
61232024-08-22T17:56:45.222ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
61242024-08-22T17:56:45.222ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
61252024-08-22T17:56:45.223ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
61262024-08-22T17:56:45.223ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
61272024-08-22T17:56:45.223ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
61282024-08-22T17:56:45.228ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
61292024-08-22T17:56:45.228ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
61302024-08-22T17:56:45.250ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 173.145094ms iteration = 16 status = {"physical_disks_added":0}
61312024-08-22T17:56:45.308ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
61322024-08-22T17:56:45.372ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1058 zone_address = fd00:1122:3344:101::2
61332024-08-22T17:56:45.372ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
61342024-08-22T17:56:45.373ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "b67afb5f-d5fb-4ec5-a071-e938cced06c6", "content-length": "26", "date": "Thu, 22 Aug 2024 17:56:44 GMT"} })
61352024-08-22T17:56:45.373ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1063 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
61362024-08-22T17:56:45.374ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1095 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
61372024-08-22T17:56:45.389ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
61382024-08-22T17:56:45.450ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1058 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 zone_address = fd00:1122:3344:101::2
61392024-08-22T17:56:45.450ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
61402024-08-22T17:56:45.450ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
61412024-08-22T17:56:45.451ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "a5546d5e-ad92-4803-9a4e-2b9ee1f99d5e", "content-length": "26", "date": "Thu, 22 Aug 2024 17:56:44 GMT"} })
61422024-08-22T17:56:45.451ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1063 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
61432024-08-22T17:56:45.451ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "62565fcd-0bea-49cd-9882-0b90a783bb4a", "content-length": "2", "date": "Thu, 22 Aug 2024 17:56:45 GMT"} })
61442024-08-22T17:56:45.452ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1095 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561
61452024-08-22T17:56:45.453ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
61462024-08-22T17:56:45.453ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
61472024-08-22T17:56:45.552ZDEBGnexus (BackgroundTasks): client request SledAgent = c961ede4-7532-4ca1-95a5-690b4d7c9b5a background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
61482024-08-22T17:56:45.553ZDEBGnexus (BackgroundTasks): client response SledAgent = c961ede4-7532-4ca1-95a5-690b4d7c9b5a background_task = vpc_route_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/vpc-routes", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "5dca72fd-ff83-4a6a-9382-96ee72f21d67", "content-length": "372", "date": "Thu, 22 Aug 2024 17:56:44 GMT"} })
61492024-08-22T17:56:45.553ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
61502024-08-22T17:56:45.553ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
61512024-08-22T17:56:45.557ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = d6f65b4e-2e87-480a-af21-a69b4520828c dns_zone_name = control-plane.oxide.internal found_so_far = 50 version = 1
61522024-08-22T17:56:45.557ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
61532024-08-22T17:56:45.557ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-08-22 17:49:57.683168 UTC dns_group = internal generation = 1
61542024-08-22T17:56:45.557ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 479.77765ms iteration = 8 status = {"generation":1}
61552024-08-22T17:56:45.557ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 480.174691ms iteration = 15 status = null
61562024-08-22T17:56:45.558ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
61572024-08-22T17:56:45.559ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
61582024-08-22T17:56:45.562ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
61592024-08-22T17:56:45.562ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
61602024-08-22T17:56:45.568ZDEBGnexus (BackgroundTasks): client request SledAgent = c961ede4-7532-4ca1-95a5-690b4d7c9b5a background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
61612024-08-22T17:56:45.568ZDEBGnexus (BackgroundTasks): client response SledAgent = c961ede4-7532-4ca1-95a5-690b4d7c9b5a background_task = vpc_route_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/vpc-routes", query: None, fragment: None }, status: 204, headers: {"x-request-id": "54e9e248-6244-42ee-8820-9b1e35036e95", "date": "Thu, 22 Aug 2024 17:56:44 GMT"} })
61622024-08-22T17:56:45.571ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 493.871332ms iteration = 15 status = {}
61632024-08-22T17:56:45.576ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
61642024-08-22T17:56:45.584ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
61652024-08-22T17:56:45.584ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 7f16ce95-fbde-4896-9a22-7f61d8d01c5d dns_zone_name = oxide.test found_so_far = 0 version = 2
61662024-08-22T17:56:45.584ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
61672024-08-22T17:56:45.584ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 507.378902ms iteration = 15 status = {}
61682024-08-22T17:56:45.585ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
61692024-08-22T17:56:45.586ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
61702024-08-22T17:56:45.589ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
61712024-08-22T17:56:45.589ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
61722024-08-22T17:56:45.589ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
61732024-08-22T17:56:45.591ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 7f16ce95-fbde-4896-9a22-7f61d8d01c5d dns_zone_name = oxide.test found_so_far = 1 version = 2
61742024-08-22T17:56:45.591ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
61752024-08-22T17:56:45.591ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-08-22 17:49:58.502032 UTC dns_group = external generation = 2
61762024-08-22T17:56:45.591ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 514.362478ms iteration = 8 status = {"generation":2}
61772024-08-22T17:56:45.593ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-08-22T17:49:43.060798Z, time_modified: 2024-08-22T17:49:43.060798Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 10d39b39-43d3-41db-b622-3ef3e09e2dad, db_silo: Silo { identity: SiloIdentity { id: 10d39b39-43d3-41db-b622-3ef3e09e2dad, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-08-22T17:49:58.442383Z, time_modified: 2024-08-22T17:49:58.442383Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "852177ddd7117113b778290a63910551c7b25a6718e9c8fda4590027c9443fee" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 10d39b39-43d3-41db-b622-3ef3e09e2dad, db_silo: Silo { identity: SiloIdentity { id: 10d39b39-43d3-41db-b622-3ef3e09e2dad, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-08-22T17:49:58.442383Z, time_modified: 2024-08-22T17:49:58.442383Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "852177ddd7117113b778290a63910551c7b25a6718e9c8fda4590027c9443fee" }] }) }
61782024-08-22T17:56:45.593ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 515.969196ms iteration = 8 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"10d39b39-43d3-41db-b622-3ef3e09e2dad","tls_certs":["852177ddd7117113b778290a63910551c7b25a6718e9c8fda4590027c9443fee"]}},"default_endpoint":{"silo_id":"10d39b39-43d3-41db-b622-3ef3e09e2dad","tls_certs":["852177ddd7117113b778290a63910551c7b25a6718e9c8fda4590027c9443fee"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
61792024-08-22T17:56:45.598ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
61802024-08-22T17:56:45.598ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
61812024-08-22T17:56:45.599ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
61822024-08-22T17:56:45.601ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
61832024-08-22T17:56:45.601ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
61842024-08-22T17:56:45.601ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
61852024-08-22T17:56:45.613ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
61862024-08-22T17:56:45.613ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::9)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "d39b8ccb-8b2c-4c0b-91d0-23115c18fd88", "date": "Thu, 22 Aug 2024 17:56:45 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
61872024-08-22T17:56:45.613ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
61882024-08-22T17:56:45.615ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::8)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "f042255c-1c67-48d2-9c83-6a77b02962c7", "date": "Thu, 22 Aug 2024 17:56:45 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
61892024-08-22T17:56:45.615ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
61902024-08-22T17:56:45.616ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
61912024-08-22T17:56:45.616ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 539.487268ms iteration = 10 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
61922024-08-22T17:56:45.633ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
61932024-08-22T17:56:45.635ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
61942024-08-22T17:56:45.635ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
61952024-08-22T17:56:45.635ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
61962024-08-22T17:56:45.657ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
61972024-08-22T17:56:45.657ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:2::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "d531dacb-4a80-4c65-a80f-3d1952392b03", "date": "Thu, 22 Aug 2024 17:56:45 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
61982024-08-22T17:56:45.657ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
61992024-08-22T17:56:45.659ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:1::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "5a50fa51-3b5b-4e02-bbea-731b1ab510f0", "date": "Thu, 22 Aug 2024 17:56:45 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
62002024-08-22T17:56:45.659ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
62012024-08-22T17:56:45.661ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:3::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "3daed0ae-9044-42cb-85b9-1f68d4e03747", "date": "Thu, 22 Aug 2024 17:56:45 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
62022024-08-22T17:56:45.661ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
62032024-08-22T17:56:45.664ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
62042024-08-22T17:56:45.664ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 586.832972ms iteration = 10 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
62052024-08-22T17:56:45.714ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
62062024-08-22T17:56:45.715ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
62072024-08-22T17:56:45.716ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:147 port = SwitchPort { id: c772d372-7448-4ce0-b8d1-efa645d18ed1, rack_id: c66d93a6-e2c3-4b19-9564-3b58bad50561, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(9ff05247-84aa-47d8-8619-ce0a8dbbaf7c) } rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 switch_location = Switch0
62082024-08-22T17:56:45.716ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
62092024-08-22T17:56:45.716ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
62102024-08-22T17:56:45.726ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
62112024-08-22T17:56:45.726ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/port/qsfp0/settings", query: Some("tag=omicron"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "252ecff6-7cf3-4b70-994e-1d26e38ae59a", "content-length": "121", "date": "Thu, 22 Aug 2024 17:56:45 GMT"} })
62122024-08-22T17:56:45.726ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1714 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G } }} } switch_port_id = "qsfp0"
62132024-08-22T17:56:45.726ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1741 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 9ff05247-84aa-47d8-8619-ce0a8dbbaf7c
62142024-08-22T17:56:45.726ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:379 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
62152024-08-22T17:56:45.726ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
62162024-08-22T17:56:45.726ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:392 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 routes = {}
62172024-08-22T17:56:45.726ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:398 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 routes = {}
62182024-08-22T17:56:45.726ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:418 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561
62192024-08-22T17:56:45.726ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
62202024-08-22T17:56:45.727ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
62212024-08-22T17:56:45.728ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
62222024-08-22T17:56:45.728ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv4", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "0cd4f1f9-3603-48f6-865c-2338ab547faa", "content-length": "2", "date": "Thu, 22 Aug 2024 17:56:45 GMT"} })
62232024-08-22T17:56:45.728ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
62242024-08-22T17:56:45.729ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "d559cf7c-4b8f-4f7f-a163-2c3461706cf9", "content-length": "52", "date": "Thu, 22 Aug 2024 17:56:45 GMT"} })
62252024-08-22T17:56:45.729ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0" }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:467 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 switch_location = Switch0
62262024-08-22T17:56:45.729ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
62272024-08-22T17:56:45.735ZDEBGnexus (BackgroundTasks): client request SledAgent = c961ede4-7532-4ca1-95a5-690b4d7c9b5a background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
62282024-08-22T17:56:45.822ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/switch-ports", query: None, fragment: None }, status: 204, headers: {"x-request-id": "42ad531e-36f6-44ac-8fa2-17b54dbeb06b", "date": "Thu, 22 Aug 2024 17:56:45 GMT"} })
62292024-08-22T17:56:45.822ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:770 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 switch = Switch0
62302024-08-22T17:56:45.822ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
62312024-08-22T17:56:45.822ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/network-bootstore-config", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "31f6025e-aaee-4f77-8c9a-d8f3214dbeae", "content-length": "667", "date": "Thu, 22 Aug 2024 17:56:45 GMT"} })
62322024-08-22T17:56:45.822ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
62332024-08-22T17:56:45.823ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
62342024-08-22T17:56:45.823ZDEBGnexus (BackgroundTasks): client response SledAgent = c961ede4-7532-4ca1-95a5-690b4d7c9b5a background_task = v2p_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/v2p", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "fea04a25-c59c-4a08-a7d2-369e0dffa6c5", "content-length": "691", "date": "Thu, 22 Aug 2024 17:56:45 GMT"} })
62352024-08-22T17:56:45.823ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = buskin
62362024-08-22T17:56:45.823ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = buskin
62372024-08-22T17:56:45.823ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = buskin
62382024-08-22T17:56:45.824ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 747.108361ms iteration = 15 status = {}
62392024-08-22T17:56:45.827ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
62402024-08-22T17:56:45.827ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
62412024-08-22T17:56:45.831ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
62422024-08-22T17:56:45.831ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
62432024-08-22T17:56:45.832ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
62442024-08-22T17:56:45.833ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
62452024-08-22T17:56:45.840ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 762.831073ms iteration = 15 status = {}
62462024-08-22T17:56:46.116ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 230 local_addr = [fd00:1122:3344:101::c]:56994 method = GET remote_addr = [fd00:1122:3344:101::d]:40670 req_id = 07ce2877-afb5-4e56-8c8e-b501484679b4 response_code = 200 uri = /eaa260db-2b19-4ae1-97e8-5735c9f5d955
62472024-08-22T17:56:55.076ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 44 reason = Timeout
62482024-08-22T17:56:55.077ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 44 reason = Timeout
62492024-08-22T17:56:55.077ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
62502024-08-22T17:56:55.077ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
62512024-08-22T17:56:55.077ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
62522024-08-22T17:56:55.077ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
62532024-08-22T17:56:55.079ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62542024-08-22T17:56:55.082ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
62552024-08-22T17:56:55.082ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
62562024-08-22T17:56:55.082ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
62572024-08-22T17:56:55.084ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
62582024-08-22T17:56:55.084ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
62592024-08-22T17:56:55.085ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62602024-08-22T17:56:55.087ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 147bdc10-344c-4ee8-a219-603cfc66e6d2, lookup_type: ById(147bdc10-344c-4ee8-a219-603cfc66e6d2) } result = Ok(())
62612024-08-22T17:56:55.087ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
62622024-08-22T17:56:55.087ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
62632024-08-22T17:56:55.094ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 147bdc10-344c-4ee8-a219-603cfc66e6d2 original_time_created = 2024-08-22 17:49:18.445237 UTC target_id = 147bdc10-344c-4ee8-a219-603cfc66e6d2 time_created = 2024-08-22 17:49:18.445237 UTC
62642024-08-22T17:56:55.094ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 17.919593ms iteration = 44 status = {"status":"target blueprint unchanged","target_id":"147bdc10-344c-4ee8-a219-603cfc66e6d2","time_created":"2024-08-22T17:49:18.445237Z"}
62652024-08-22T17:56:55.102ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
62662024-08-22T17:56:55.102ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
62672024-08-22T17:56:55.126ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 49.867538ms iteration = 44 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
62682024-08-22T17:56:56.117ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 198 local_addr = [fd00:1122:3344:101::c]:56994 method = GET remote_addr = [fd00:1122:3344:101::d]:40670 req_id = ba7ab370-5ee8-4455-a8e6-4b7d714c9527 response_code = 200 uri = /eaa260db-2b19-4ae1-97e8-5735c9f5d955
62692024-08-22T17:57:05.077ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 45 reason = Timeout
62702024-08-22T17:57:05.077ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
62712024-08-22T17:57:05.077ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 45 reason = Timeout
62722024-08-22T17:57:05.077ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
62732024-08-22T17:57:05.077ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
62742024-08-22T17:57:05.078ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
62752024-08-22T17:57:05.079ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
62762024-08-22T17:57:05.079ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62772024-08-22T17:57:05.080ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
62782024-08-22T17:57:05.082ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
62792024-08-22T17:57:05.082ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
62802024-08-22T17:57:05.082ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
62812024-08-22T17:57:05.084ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
62822024-08-22T17:57:05.084ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
62832024-08-22T17:57:05.085ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62842024-08-22T17:57:05.088ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 147bdc10-344c-4ee8-a219-603cfc66e6d2, lookup_type: ById(147bdc10-344c-4ee8-a219-603cfc66e6d2) } result = Ok(())
62852024-08-22T17:57:05.088ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
62862024-08-22T17:57:05.088ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
62872024-08-22T17:57:05.097ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 147bdc10-344c-4ee8-a219-603cfc66e6d2 original_time_created = 2024-08-22 17:49:18.445237 UTC target_id = 147bdc10-344c-4ee8-a219-603cfc66e6d2 time_created = 2024-08-22 17:49:18.445237 UTC
62882024-08-22T17:57:05.098ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 20.425095ms iteration = 45 status = {"status":"target blueprint unchanged","target_id":"147bdc10-344c-4ee8-a219-603cfc66e6d2","time_created":"2024-08-22T17:49:18.445237Z"}
62892024-08-22T17:57:05.111ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 34.042286ms iteration = 45 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
62902024-08-22T17:57:06.117ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 304 local_addr = [fd00:1122:3344:101::c]:56994 method = GET remote_addr = [fd00:1122:3344:101::d]:40670 req_id = c08de5f5-7e25-442c-866b-0fa334459aca response_code = 200 uri = /eaa260db-2b19-4ae1-97e8-5735c9f5d955
62912024-08-22T17:57:15.078ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 46 reason = Timeout
62922024-08-22T17:57:15.078ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
62932024-08-22T17:57:15.078ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 16 reason = Timeout
62942024-08-22T17:57:15.078ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 16 reason = Timeout
62952024-08-22T17:57:15.078ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
62962024-08-22T17:57:15.078ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
62972024-08-22T17:57:15.078ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
62982024-08-22T17:57:15.078ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 16 reason = Timeout
62992024-08-22T17:57:15.078ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
63002024-08-22T17:57:15.078ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 16 reason = Timeout
63012024-08-22T17:57:15.078ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
63022024-08-22T17:57:15.078ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 16 reason = Timeout
63032024-08-22T17:57:15.078ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 16 reason = Timeout
63042024-08-22T17:57:15.078ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
63052024-08-22T17:57:15.078ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
63062024-08-22T17:57:15.078ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 16 reason = Timeout
63072024-08-22T17:57:15.078ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
63082024-08-22T17:57:15.078ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 16 reason = Timeout
63092024-08-22T17:57:15.078ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
63102024-08-22T17:57:15.078ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 46 reason = Timeout
63112024-08-22T17:57:15.078ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 16 reason = Timeout
63122024-08-22T17:57:15.078ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
63132024-08-22T17:57:15.078ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
63142024-08-22T17:57:15.078ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 16 reason = Timeout
63152024-08-22T17:57:15.078ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
63162024-08-22T17:57:15.078ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 16 reason = Timeout
63172024-08-22T17:57:15.078ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 17 reason = Timeout
63182024-08-22T17:57:15.078ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
63192024-08-22T17:57:15.078ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
63202024-08-22T17:57:15.078ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
63212024-08-22T17:57:15.078ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
63222024-08-22T17:57:15.078ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
63232024-08-22T17:57:15.078ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
63242024-08-22T17:57:15.078ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
63252024-08-22T17:57:15.078ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
63262024-08-22T17:57:15.078ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
63272024-08-22T17:57:15.078ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
63282024-08-22T17:57:15.078ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
63292024-08-22T17:57:15.078ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
63302024-08-22T17:57:15.078ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
63312024-08-22T17:57:15.078ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
63322024-08-22T17:57:15.079ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 16 reason = Timeout
63332024-08-22T17:57:15.079ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
63342024-08-22T17:57:15.079ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
63352024-08-22T17:57:15.080ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
63362024-08-22T17:57:15.081ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
63372024-08-22T17:57:15.081ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
63382024-08-22T17:57:15.081ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.259677ms iteration = 16 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
63392024-08-22T17:57:15.083ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
63402024-08-22T17:57:15.083ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
63412024-08-22T17:57:15.083ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
63422024-08-22T17:57:15.083ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
63432024-08-22T17:57:15.083ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
63442024-08-22T17:57:15.083ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
63452024-08-22T17:57:15.083ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
63462024-08-22T17:57:15.083ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
63472024-08-22T17:57:15.083ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
63482024-08-22T17:57:15.086ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-c961ede4-7532-4ca1-95a5-690b4d7c9b5a.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-c961ede4-7532-4ca1-95a5-690b4d7c9b5a.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 2023, tv_nsec: 610097118 } })
63492024-08-22T17:57:15.088ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1050
63502024-08-22T17:57:15.089ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
63512024-08-22T17:57:15.090ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
63522024-08-22T17:57:15.091ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-c961ede4-7532-4ca1-95a5-690b4d7c9b5a.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-c961ede4-7532-4ca1-95a5-690b4d7c9b5a.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 2023, tv_nsec: 615013343 } })
63532024-08-22T17:57:15.091ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 12.307083ms iteration = 16 status = {"errors":[],"garbage_collect_requested":[]}
63542024-08-22T17:57:15.091ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1050
63552024-08-22T17:57:15.092ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
63562024-08-22T17:57:15.092ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
63572024-08-22T17:57:15.092ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
63582024-08-22T17:57:15.092ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
63592024-08-22T17:57:15.092ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
63602024-08-22T17:57:15.092ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
63612024-08-22T17:57:15.097ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
63622024-08-22T17:57:15.098ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
63632024-08-22T17:57:15.098ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-c961ede4-7532-4ca1-95a5-690b4d7c9b5a.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-c961ede4-7532-4ca1-95a5-690b4d7c9b5a.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 2023, tv_nsec: 622066860 } })
63642024-08-22T17:57:15.098ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
63652024-08-22T17:57:15.098ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1050 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561
63662024-08-22T17:57:15.098ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
63672024-08-22T17:57:15.099ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
63682024-08-22T17:57:15.101ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
63692024-08-22T17:57:15.101ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
63702024-08-22T17:57:15.101ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
63712024-08-22T17:57:15.123ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
63722024-08-22T17:57:15.123ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
63732024-08-22T17:57:15.124ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
63742024-08-22T17:57:15.124ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:55
63752024-08-22T17:57:15.124ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
63762024-08-22T17:57:15.124ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
63772024-08-22T17:57:15.125ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
63782024-08-22T17:57:15.128ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
63792024-08-22T17:57:15.128ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
63802024-08-22T17:57:15.128ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
63812024-08-22T17:57:15.132ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
63822024-08-22T17:57:15.133ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
63832024-08-22T17:57:15.134ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
63842024-08-22T17:57:15.137ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 147bdc10-344c-4ee8-a219-603cfc66e6d2, lookup_type: ById(147bdc10-344c-4ee8-a219-603cfc66e6d2) } result = Ok(())
63852024-08-22T17:57:15.137ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
63862024-08-22T17:57:15.138ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
63872024-08-22T17:57:15.138ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
63882024-08-22T17:57:15.139ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
63892024-08-22T17:57:15.139ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
63902024-08-22T17:57:15.140ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
63912024-08-22T17:57:15.140ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
63922024-08-22T17:57:15.141ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
63932024-08-22T17:57:15.141ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
63942024-08-22T17:57:15.141ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
63952024-08-22T17:57:15.142ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
63962024-08-22T17:57:15.142ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
63972024-08-22T17:57:15.142ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 64.636504ms iteration = 16 status = {"region_replacement_started_err":0,"region_replacement_started_ok":0}
63982024-08-22T17:57:15.143ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
63992024-08-22T17:57:15.144ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
64002024-08-22T17:57:15.144ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
64012024-08-22T17:57:15.147ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: c961ede4-7532-4ca1-95a5-690b4d7c9b5a, lookup_type: ById(c961ede4-7532-4ca1-95a5-690b4d7c9b5a) } result = Ok(())
64022024-08-22T17:57:15.147ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
64032024-08-22T17:57:15.147ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
64042024-08-22T17:57:15.148ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 147bdc10-344c-4ee8-a219-603cfc66e6d2 original_time_created = 2024-08-22 17:49:18.445237 UTC target_id = 147bdc10-344c-4ee8-a219-603cfc66e6d2 time_created = 2024-08-22 17:49:18.445237 UTC
64052024-08-22T17:57:15.148ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 70.255433ms iteration = 46 status = {"status":"target blueprint unchanged","target_id":"147bdc10-344c-4ee8-a219-603cfc66e6d2","time_created":"2024-08-22T17:49:18.445237Z"}
64062024-08-22T17:57:15.149ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 71.57449ms iteration = 16 status = 0
64072024-08-22T17:57:15.151ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1058 zone_address = fd00:1122:3344:101::2
64082024-08-22T17:57:15.151ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
64092024-08-22T17:57:15.152ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "bc44f8e1-f4dd-4c29-b8db-11f5e597f1bc", "content-length": "26", "date": "Thu, 22 Aug 2024 17:57:14 GMT"} })
64102024-08-22T17:57:15.152ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1063 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
64112024-08-22T17:57:15.152ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1058 zone_address = fd00:1122:3344:101::2
64122024-08-22T17:57:15.152ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
64132024-08-22T17:57:15.152ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "2b0b209b-8ab0-460d-9075-19a69feb91df", "content-length": "26", "date": "Thu, 22 Aug 2024 17:57:14 GMT"} })
64142024-08-22T17:57:15.152ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1063 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
64152024-08-22T17:57:15.153ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1095 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
64162024-08-22T17:57:15.154ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1095 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
64172024-08-22T17:57:15.155ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 77.092659ms iteration = 46 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
64182024-08-22T17:57:15.162ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:55
64192024-08-22T17:57:15.162ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:220 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
64202024-08-22T17:57:15.162ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 84.605468ms iteration = 16 status = {"destroyed_active_vmms":0,"error":null,"saga_start_failures":0,"sagas_completed":0,"sagas_failed":0,"sagas_started":0,"terminated_active_migrations":0}
64212024-08-22T17:57:15.168ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:454 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
64222024-08-22T17:57:15.168ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 90.243456ms iteration = 16 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
64232024-08-22T17:57:15.169ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1058 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 zone_address = fd00:1122:3344:101::2
64242024-08-22T17:57:15.169ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
64252024-08-22T17:57:15.171ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "003c1d57-e412-49e9-bd07-70bdbd5f7220", "content-length": "26", "date": "Thu, 22 Aug 2024 17:57:14 GMT"} })
64262024-08-22T17:57:15.171ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1063 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
64272024-08-22T17:57:15.173ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1095 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561
64282024-08-22T17:57:15.187ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 109.293595ms iteration = 16 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
64292024-08-22T17:57:15.207ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
64302024-08-22T17:57:15.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
64312024-08-22T17:57:15.228ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 150.248146ms iteration = 17 status = {"physical_disks_added":0}
64322024-08-22T17:57:15.255ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 176.890323ms iteration = 16 status = {}
64332024-08-22T17:57:15.263ZDEBGnexus (BackgroundTasks): client request SledAgent = c961ede4-7532-4ca1-95a5-690b4d7c9b5a background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
64342024-08-22T17:57:15.263ZDEBGnexus (BackgroundTasks): client response SledAgent = c961ede4-7532-4ca1-95a5-690b4d7c9b5a background_task = vpc_route_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/vpc-routes", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "f24a6730-7195-4206-a9cf-9f883987ad1f", "content-length": "372", "date": "Thu, 22 Aug 2024 17:57:15 GMT"} })
64352024-08-22T17:57:15.263ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
64362024-08-22T17:57:15.264ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
64372024-08-22T17:57:15.265ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
64382024-08-22T17:57:15.266ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
64392024-08-22T17:57:15.267ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
64402024-08-22T17:57:15.267ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "5642323b-db0e-443c-afd0-e166317f5c82", "content-length": "2", "date": "Thu, 22 Aug 2024 17:57:14 GMT"} })
64412024-08-22T17:57:15.268ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
64422024-08-22T17:57:15.268ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
64432024-08-22T17:57:15.268ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
64442024-08-22T17:57:15.269ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
64452024-08-22T17:57:15.270ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 192.430303ms iteration = 16 status = null
64462024-08-22T17:57:15.274ZDEBGnexus (BackgroundTasks): client request SledAgent = c961ede4-7532-4ca1-95a5-690b4d7c9b5a background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
64472024-08-22T17:57:15.274ZDEBGnexus (BackgroundTasks): client response SledAgent = c961ede4-7532-4ca1-95a5-690b4d7c9b5a background_task = vpc_route_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/vpc-routes", query: None, fragment: None }, status: 204, headers: {"x-request-id": "be5eeb7a-39a2-4e17-b18f-0918d31878f6", "date": "Thu, 22 Aug 2024 17:57:15 GMT"} })
64482024-08-22T17:57:15.276ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 198.461244ms iteration = 16 status = {}
64492024-08-22T17:57:15.333ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
64502024-08-22T17:57:15.334ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
64512024-08-22T17:57:15.336ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:147 port = SwitchPort { id: c772d372-7448-4ce0-b8d1-efa645d18ed1, rack_id: c66d93a6-e2c3-4b19-9564-3b58bad50561, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(9ff05247-84aa-47d8-8619-ce0a8dbbaf7c) } rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 switch_location = Switch0
64522024-08-22T17:57:15.336ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
64532024-08-22T17:57:15.336ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
64542024-08-22T17:57:15.346ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
64552024-08-22T17:57:15.346ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
64562024-08-22T17:57:15.346ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
64572024-08-22T17:57:15.347ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/port/qsfp0/settings", query: Some("tag=omicron"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "9f44f0aa-f303-48af-908f-e0756a03135e", "content-length": "121", "date": "Thu, 22 Aug 2024 17:57:14 GMT"} })
64582024-08-22T17:57:15.347ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1714 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G } }} } switch_port_id = "qsfp0"
64592024-08-22T17:57:15.347ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1741 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 9ff05247-84aa-47d8-8619-ce0a8dbbaf7c
64602024-08-22T17:57:15.347ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:379 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
64612024-08-22T17:57:15.347ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
64622024-08-22T17:57:15.347ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:392 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 routes = {}
64632024-08-22T17:57:15.347ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:398 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 routes = {}
64642024-08-22T17:57:15.347ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:418 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561
64652024-08-22T17:57:15.347ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
64662024-08-22T17:57:15.347ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
64672024-08-22T17:57:15.348ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
64682024-08-22T17:57:15.349ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
64692024-08-22T17:57:15.349ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv4", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "38281fd9-dbb9-47a1-99a6-edcb38156e1b", "content-length": "2", "date": "Thu, 22 Aug 2024 17:57:14 GMT"} })
64702024-08-22T17:57:15.349ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
64712024-08-22T17:57:15.350ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "2c6d040b-5589-4748-8913-dc35afb3073c", "content-length": "52", "date": "Thu, 22 Aug 2024 17:57:14 GMT"} })
64722024-08-22T17:57:15.350ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0" }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:467 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 switch_location = Switch0
64732024-08-22T17:57:15.350ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
64742024-08-22T17:57:15.350ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
64752024-08-22T17:57:15.350ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
64762024-08-22T17:57:15.350ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
64772024-08-22T17:57:15.377ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
64782024-08-22T17:57:15.379ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
64792024-08-22T17:57:15.379ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
64802024-08-22T17:57:15.379ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
64812024-08-22T17:57:15.413ZDEBGnexus (BackgroundTasks): client request SledAgent = c961ede4-7532-4ca1-95a5-690b4d7c9b5a background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
64822024-08-22T17:57:15.442ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/switch-ports", query: None, fragment: None }, status: 204, headers: {"x-request-id": "e761d78f-2a29-4949-a337-a6d7e05549af", "date": "Thu, 22 Aug 2024 17:57:15 GMT"} })
64832024-08-22T17:57:15.442ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:770 rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 switch = Switch0
64842024-08-22T17:57:15.442ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
64852024-08-22T17:57:15.442ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = c66d93a6-e2c3-4b19-9564-3b58bad50561 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/network-bootstore-config", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "26f9f2c3-b1dd-4462-bbe1-e883b4b78ffc", "content-length": "667", "date": "Thu, 22 Aug 2024 17:57:15 GMT"} })
64862024-08-22T17:57:15.442ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
64872024-08-22T17:57:15.442ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
64882024-08-22T17:57:15.442ZDEBGnexus (BackgroundTasks): client response SledAgent = c961ede4-7532-4ca1-95a5-690b4d7c9b5a background_task = v2p_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/v2p", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "021c7c2b-fc8e-4a08-9a76-64f27cbc7f85", "content-length": "691", "date": "Thu, 22 Aug 2024 17:57:15 GMT"} })
64892024-08-22T17:57:15.442ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = buskin
64902024-08-22T17:57:15.442ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = buskin
64912024-08-22T17:57:15.442ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = buskin
64922024-08-22T17:57:15.444ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 366.169191ms iteration = 16 status = {}
64932024-08-22T17:57:15.446ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
64942024-08-22T17:57:15.446ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
64952024-08-22T17:57:15.449ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
64962024-08-22T17:57:15.450ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
64972024-08-22T17:57:15.451ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
64982024-08-22T17:57:15.452ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
64992024-08-22T17:57:15.458ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 380.641206ms iteration = 16 status = {}
65002024-08-22T17:57:16.117ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 225 local_addr = [fd00:1122:3344:101::c]:56994 method = GET remote_addr = [fd00:1122:3344:101::d]:40670 req_id = fb6025bf-a0b6-4ce4-ac75-1715fcdfd484 response_code = 200 uri = /eaa260db-2b19-4ae1-97e8-5735c9f5d955
65012024-08-22T17:57:25.077ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 47 reason = Timeout
65022024-08-22T17:57:25.077ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 47 reason = Timeout
65032024-08-22T17:57:25.077ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
65042024-08-22T17:57:25.077ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
65052024-08-22T17:57:25.077ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
65062024-08-22T17:57:25.077ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
65072024-08-22T17:57:25.080ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
65082024-08-22T17:57:25.080ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
65092024-08-22T17:57:25.105ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
65102024-08-22T17:57:25.108ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
65112024-08-22T17:57:25.108ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
65122024-08-22T17:57:25.108ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
65132024-08-22T17:57:25.110ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
65142024-08-22T17:57:25.110ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
65152024-08-22T17:57:25.112ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
65162024-08-22T17:57:25.112ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 35.314782ms iteration = 47 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
65172024-08-22T17:57:25.114ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 147bdc10-344c-4ee8-a219-603cfc66e6d2, lookup_type: ById(147bdc10-344c-4ee8-a219-603cfc66e6d2) } result = Ok(())
65182024-08-22T17:57:25.114ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
65192024-08-22T17:57:25.114ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
65202024-08-22T17:57:25.122ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 147bdc10-344c-4ee8-a219-603cfc66e6d2 original_time_created = 2024-08-22 17:49:18.445237 UTC target_id = 147bdc10-344c-4ee8-a219-603cfc66e6d2 time_created = 2024-08-22 17:49:18.445237 UTC
65212024-08-22T17:57:25.122ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 45.451906ms iteration = 47 status = {"status":"target blueprint unchanged","target_id":"147bdc10-344c-4ee8-a219-603cfc66e6d2","time_created":"2024-08-22T17:49:18.445237Z"}
65222024-08-22T17:57:26.117ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 228 local_addr = [fd00:1122:3344:101::c]:56994 method = GET remote_addr = [fd00:1122:3344:101::d]:40670 req_id = c42363a6-4b29-4ec0-acf4-be0c84837b38 response_code = 200 uri = /eaa260db-2b19-4ae1-97e8-5735c9f5d955
65232024-08-22T17:57:35.078ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 48 reason = Timeout
65242024-08-22T17:57:35.078ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 48 reason = Timeout
65252024-08-22T17:57:35.078ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
65262024-08-22T17:57:35.078ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
65272024-08-22T17:57:35.078ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
65282024-08-22T17:57:35.078ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
65292024-08-22T17:57:35.083ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
65302024-08-22T17:57:35.086ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
65312024-08-22T17:57:35.086ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
65322024-08-22T17:57:35.086ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
65332024-08-22T17:57:35.088ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
65342024-08-22T17:57:35.088ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
65352024-08-22T17:57:35.109ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
65362024-08-22T17:57:35.110ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
65372024-08-22T17:57:35.111ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 32.79789ms iteration = 48 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
65382024-08-22T17:57:35.114ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
65392024-08-22T17:57:35.117ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 147bdc10-344c-4ee8-a219-603cfc66e6d2, lookup_type: ById(147bdc10-344c-4ee8-a219-603cfc66e6d2) } result = Ok(())
65402024-08-22T17:57:35.117ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
65412024-08-22T17:57:35.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
65422024-08-22T17:57:35.125ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 147bdc10-344c-4ee8-a219-603cfc66e6d2 original_time_created = 2024-08-22 17:49:18.445237 UTC target_id = 147bdc10-344c-4ee8-a219-603cfc66e6d2 time_created = 2024-08-22 17:49:18.445237 UTC
65432024-08-22T17:57:35.125ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 47.321215ms iteration = 48 status = {"status":"target blueprint unchanged","target_id":"147bdc10-344c-4ee8-a219-603cfc66e6d2","time_created":"2024-08-22T17:49:18.445237Z"}
65442024-08-22T17:57:36.118ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/06c8dab/dropshot/src/server.rs:914 latency_us = 805 local_addr = [fd00:1122:3344:101::c]:56994 method = GET remote_addr = [fd00:1122:3344:101::d]:40670 req_id = 10ece263-3270-4362-9cf6-a16d213be841 response_code = 200 uri = /eaa260db-2b19-4ae1-97e8-5735c9f5d955