Max level shown:
1 [ Jun 3 00:50:57 Enabled. ]
2 [ Jun 3 00:50:57 Rereading configuration. ]
3 [ Jun 3 00:51:00 Rereading configuration. ]
4 [ Jun 3 00:51:06 Executing start method ("ctrun -l child -o noorphan,regent /opt/oxide/omicron-nexus/bin/nexus /var/svc/manifest/site/nexus/config.toml &"). ]
5 [ Jun 3 00:51:06 Method "start" exited with status 0. ]
6 note: configured to log to "/dev/stdout"
72025-06-03T00:51:06.560ZDEBGnexus: registered DTrace probes
82025-06-03T00:51:06.562ZINFOnexus: setting up nexus server file = nexus/src/lib.rs:78
92025-06-03T00:51:06.572ZINFOnexus (ServerContext): registering Oso class class = Action file = nexus/auth/src/authz/oso_generic.rs:68
102025-06-03T00:51:06.572ZINFOnexus (ServerContext): registering Oso class class = AnyActor file = nexus/auth/src/authz/oso_generic.rs:68
112025-06-03T00:51:06.572ZINFOnexus (ServerContext): registering Oso class class = AuthenticatedActor file = nexus/auth/src/authz/oso_generic.rs:68
122025-06-03T00:51:06.572ZINFOnexus (ServerContext): registering Oso class class = BlueprintConfig file = nexus/auth/src/authz/oso_generic.rs:68
132025-06-03T00:51:06.572ZINFOnexus (ServerContext): registering Oso class class = Database file = nexus/auth/src/authz/oso_generic.rs:68
142025-06-03T00:51:06.572ZINFOnexus (ServerContext): registering Oso class class = DnsConfig file = nexus/auth/src/authz/oso_generic.rs:68
152025-06-03T00:51:06.572ZINFOnexus (ServerContext): registering Oso class class = Fleet file = nexus/auth/src/authz/oso_generic.rs:68
162025-06-03T00:51:06.572ZINFOnexus (ServerContext): registering Oso class class = Inventory file = nexus/auth/src/authz/oso_generic.rs:68
172025-06-03T00:51:06.572ZINFOnexus (ServerContext): registering Oso class class = IpPoolList file = nexus/auth/src/authz/oso_generic.rs:68
182025-06-03T00:51:06.572ZINFOnexus (ServerContext): registering Oso class class = ConsoleSessionList file = nexus/auth/src/authz/oso_generic.rs:68
192025-06-03T00:51:06.572ZINFOnexus (ServerContext): registering Oso class class = DeviceAuthRequestList file = nexus/auth/src/authz/oso_generic.rs:68
202025-06-03T00:51:06.572ZINFOnexus (ServerContext): registering Oso class class = SiloCertificateList file = nexus/auth/src/authz/oso_generic.rs:68
212025-06-03T00:51:06.572ZINFOnexus (ServerContext): registering Oso class class = SiloIdentityProviderList file = nexus/auth/src/authz/oso_generic.rs:68
222025-06-03T00:51:06.572ZINFOnexus (ServerContext): registering Oso class class = SiloUserList file = nexus/auth/src/authz/oso_generic.rs:68
232025-06-03T00:51:06.572ZINFOnexus (ServerContext): registering Oso class class = TargetReleaseConfig file = nexus/auth/src/authz/oso_generic.rs:68
242025-06-03T00:51:06.572ZINFOnexus (ServerContext): registering Oso class class = AlertClassList file = nexus/auth/src/authz/oso_generic.rs:68
252025-06-03T00:51:06.572ZINFOnexus (ServerContext): registering Oso class class = Project file = nexus/auth/src/authz/oso_generic.rs:68
262025-06-03T00:51:06.572ZINFOnexus (ServerContext): registering Oso class class = Disk file = nexus/auth/src/authz/oso_generic.rs:68
272025-06-03T00:51:06.572ZINFOnexus (ServerContext): registering Oso class class = Snapshot file = nexus/auth/src/authz/oso_generic.rs:68
282025-06-03T00:51:06.572ZINFOnexus (ServerContext): registering Oso class class = ProjectImage file = nexus/auth/src/authz/oso_generic.rs:68
292025-06-03T00:51:06.572ZINFOnexus (ServerContext): registering Oso class class = AffinityGroup file = nexus/auth/src/authz/oso_generic.rs:68
302025-06-03T00:51:06.572ZINFOnexus (ServerContext): registering Oso class class = AntiAffinityGroup file = nexus/auth/src/authz/oso_generic.rs:68
312025-06-03T00:51:06.572ZINFOnexus (ServerContext): registering Oso class class = Instance file = nexus/auth/src/authz/oso_generic.rs:68
322025-06-03T00:51:06.572ZINFOnexus (ServerContext): registering Oso class class = IpPool file = nexus/auth/src/authz/oso_generic.rs:68
332025-06-03T00:51:06.572ZINFOnexus (ServerContext): registering Oso class class = InstanceNetworkInterface file = nexus/auth/src/authz/oso_generic.rs:68
342025-06-03T00:51:06.572ZINFOnexus (ServerContext): registering Oso class class = Vpc file = nexus/auth/src/authz/oso_generic.rs:68
352025-06-03T00:51:06.572ZINFOnexus (ServerContext): registering Oso class class = VpcRouter file = nexus/auth/src/authz/oso_generic.rs:68
362025-06-03T00:51:06.572ZINFOnexus (ServerContext): registering Oso class class = InternetGateway file = nexus/auth/src/authz/oso_generic.rs:68
372025-06-03T00:51:06.572ZINFOnexus (ServerContext): registering Oso class class = InternetGatewayIpPool file = nexus/auth/src/authz/oso_generic.rs:68
382025-06-03T00:51:06.572ZINFOnexus (ServerContext): registering Oso class class = InternetGatewayIpAddress file = nexus/auth/src/authz/oso_generic.rs:68
392025-06-03T00:51:06.572ZINFOnexus (ServerContext): registering Oso class class = RouterRoute file = nexus/auth/src/authz/oso_generic.rs:68
402025-06-03T00:51:06.572ZINFOnexus (ServerContext): registering Oso class class = VpcSubnet file = nexus/auth/src/authz/oso_generic.rs:68
412025-06-03T00:51:06.572ZINFOnexus (ServerContext): registering Oso class class = FloatingIp file = nexus/auth/src/authz/oso_generic.rs:68
422025-06-03T00:51:06.572ZINFOnexus (ServerContext): registering Oso class class = Image file = nexus/auth/src/authz/oso_generic.rs:68
432025-06-03T00:51:06.572ZINFOnexus (ServerContext): registering Oso class class = SiloImage file = nexus/auth/src/authz/oso_generic.rs:68
442025-06-03T00:51:06.572ZINFOnexus (ServerContext): registering Oso class class = AddressLot file = nexus/auth/src/authz/oso_generic.rs:68
452025-06-03T00:51:06.572ZINFOnexus (ServerContext): registering Oso class class = Blueprint file = nexus/auth/src/authz/oso_generic.rs:68
462025-06-03T00:51:06.572ZINFOnexus (ServerContext): registering Oso class class = LoopbackAddress file = nexus/auth/src/authz/oso_generic.rs:68
472025-06-03T00:51:06.572ZINFOnexus (ServerContext): registering Oso class class = Certificate file = nexus/auth/src/authz/oso_generic.rs:68
482025-06-03T00:51:06.572ZINFOnexus (ServerContext): registering Oso class class = ConsoleSession file = nexus/auth/src/authz/oso_generic.rs:68
492025-06-03T00:51:06.572ZINFOnexus (ServerContext): registering Oso class class = DeviceAuthRequest file = nexus/auth/src/authz/oso_generic.rs:68
502025-06-03T00:51:06.572ZINFOnexus (ServerContext): registering Oso class class = DeviceAccessToken file = nexus/auth/src/authz/oso_generic.rs:68
512025-06-03T00:51:06.572ZINFOnexus (ServerContext): registering Oso class class = PhysicalDisk file = nexus/auth/src/authz/oso_generic.rs:68
522025-06-03T00:51:06.572ZINFOnexus (ServerContext): registering Oso class class = Rack file = nexus/auth/src/authz/oso_generic.rs:68
532025-06-03T00:51:06.572ZINFOnexus (ServerContext): registering Oso class class = RoleBuiltin file = nexus/auth/src/authz/oso_generic.rs:68
542025-06-03T00:51:06.572ZINFOnexus (ServerContext): registering Oso class class = SshKey file = nexus/auth/src/authz/oso_generic.rs:68
552025-06-03T00:51:06.572ZINFOnexus (ServerContext): registering Oso class class = Silo file = nexus/auth/src/authz/oso_generic.rs:68
562025-06-03T00:51:06.572ZINFOnexus (ServerContext): registering Oso class class = SiloUser file = nexus/auth/src/authz/oso_generic.rs:68
572025-06-03T00:51:06.572ZINFOnexus (ServerContext): registering Oso class class = SiloGroup file = nexus/auth/src/authz/oso_generic.rs:68
582025-06-03T00:51:06.572ZINFOnexus (ServerContext): registering Oso class class = SupportBundle file = nexus/auth/src/authz/oso_generic.rs:68
592025-06-03T00:51:06.572ZINFOnexus (ServerContext): registering Oso class class = IdentityProvider file = nexus/auth/src/authz/oso_generic.rs:68
602025-06-03T00:51:06.572ZINFOnexus (ServerContext): registering Oso class class = SamlIdentityProvider file = nexus/auth/src/authz/oso_generic.rs:68
612025-06-03T00:51:06.572ZINFOnexus (ServerContext): registering Oso class class = Sled file = nexus/auth/src/authz/oso_generic.rs:68
622025-06-03T00:51:06.572ZINFOnexus (ServerContext): registering Oso class class = TufRepo file = nexus/auth/src/authz/oso_generic.rs:68
632025-06-03T00:51:06.572ZINFOnexus (ServerContext): registering Oso class class = TufArtifact file = nexus/auth/src/authz/oso_generic.rs:68
642025-06-03T00:51:06.572ZINFOnexus (ServerContext): registering Oso class class = Alert file = nexus/auth/src/authz/oso_generic.rs:68
652025-06-03T00:51:06.572ZINFOnexus (ServerContext): registering Oso class class = AlertReceiver file = nexus/auth/src/authz/oso_generic.rs:68
662025-06-03T00:51:06.572ZINFOnexus (ServerContext): registering Oso class class = WebhookSecret file = nexus/auth/src/authz/oso_generic.rs:68
672025-06-03T00:51:06.572ZINFOnexus (ServerContext): registering Oso class class = Zpool file = nexus/auth/src/authz/oso_generic.rs:68
682025-06-03T00:51:06.572ZINFOnexus (ServerContext): registering Oso class class = Service file = nexus/auth/src/authz/oso_generic.rs:68
692025-06-03T00:51:06.572ZINFOnexus (ServerContext): registering Oso class class = UserBuiltin file = nexus/auth/src/authz/oso_generic.rs:68
702025-06-03T00:51:06.573ZINFOnexus (ServerContext): full Oso configuration config = #\n# Oso configuration for Omicron\n# This file is augmented by generated snippets.\n#\n\n#\n# ACTOR TYPES AND BASIC RULES\n#\n\n# `AnyActor` includes both authenticated and unauthenticated users.\nactor AnyActor {}\n\n# An `AuthenticatedActor` has an identity in the system. All of our operations\n# today require that an actor be authenticated.\nactor AuthenticatedActor {}\n\n# For any resource, `actor` can perform action `action` on it if they're\n# authenticated and their role(s) give them the corresponding permission on that\n# resource.\nallow(actor: AnyActor, action: Action, resource) if\n actor.authenticated and\n has_permission(actor.authn_actor.unwrap(), action.to_perm(), resource);\n\n# Define role relationships\nhas_role(actor: AuthenticatedActor, role: String, resource: Resource)\n\tif resource.has_role(actor, role);\n\n#\n# ROLES AND PERMISSIONS IN THE FLEET/SILO/PROJECT HIERARCHY\n#\n# We define the following permissions for most resources in the system:\n#\n# - "create_child": required to create child resources (of any type)\n#\n# - "list_children": required to list child resources (of all types) of a\n# resource\n#\n# - "modify": required to modify or delete a resource\n#\n# - "read": required to read a resource\n#\n# We define the following predefined roles for only a few high-level resources:\n# the Fleet (see below), Silo, Organization, and Project. The specific roles\n# are oriented around intended use-cases:\n#\n# - "admin": has all permissions on the resource\n#\n# - "collaborator": has "read", "list_children", and "create_child", plus\n# the "admin" role for child resources. The idea is that if you're an\n# Organization Collaborator, you have full control over the Projects within\n# the Organization, but you cannot modify or delete the Organization itself.\n#\n# - "viewer": has "read" and "list_children" on a resource\n#\n# Below the Project level, permissions are granted via roles at the Project\n# level. For example, for someone to be able to create, modify, or delete any\n# Instances, they must be granted project.collaborator, which means they can\n# create, modify, or delete _all_ resources in the Project.\n#\n# The complete set of predefined roles:\n#\n# - fleet.admin (superuser for the whole system)\n# - fleet.collaborator (can manage Silos)\n# - fleet.viewer (can read most non-siloed resources in the system)\n# - silo.admin (superuser for the silo)\n# - silo.collaborator (can create and own Organizations)\n# - silo.viewer (can read most resources within the Silo)\n# - organization.admin (complete control over an organization)\n# - organization.collaborator (can manage Projects)\n# - organization.viewer (can read most resources within the Organization)\n# - project.admin (complete control over a Project)\n# - project.collaborator (can manage all resources within the Project)\n# - project.viewer (can read most resources within the Project)\n#\n# Outside the Silo/Organization/Project hierarchy, we (currently) treat most\n# resources as nested under Fleet or else a synthetic resource (see below). We\n# do not yet support role assignments on anything other than Fleet, Silo,\n# Organization, or Project.\n#\n\n# "Fleet" is a global singleton representing the whole system. The name comes\n# from the idea described in RFD 24, but it's not quite right. This probably\n# should be more like "Region" or "AvailabilityZone". The precise boundaries\n# have not yet been figured out.\nresource Fleet {\n\tpermissions = [\n\t "list_children",\n\t "modify",\n\t "read",\n\t "create_child",\n\t];\n\n\troles = [\n\t # Roles that can be attached by users\n\t "admin",\n\t "collaborator",\n\t "viewer",\n\n\t # Internal-only roles\n\t "external-authenticator"\n\t];\n\n\t# Roles implied by other roles on this resource\n\t"viewer" if "collaborator";\n\t"collaborator" if "admin";\n\n\t# Permissions granted directly by roles on this resource\n\t"list_children" if "viewer";\n\t"read" if "viewer";\n\t"create_child" if "collaborator";\n\t"modify" if "admin";\n}\n\n# For fleets specifically, roles can be conferred by roles on the user's Silo.\nhas_role(actor: AuthenticatedActor, role: String, _: Fleet) if\n\tsilo_role in actor.confers_fleet_role(role) and\n\thas_role(actor, silo_role, actor.silo.unwrap());\n\nresource Silo {\n\tpermissions = [\n\t "list_children",\n\t "modify",\n\t "read",\n\t "create_child",\n\t];\n\troles = [ "admin", "collaborator", "viewer" ];\n\n\t# Roles implied by other roles on this resource\n\t"viewer" if "collaborator";\n\t"collaborator" if "admin";\n\n\t# Permissions granted directly by roles on this resource\n\t"list_children" if "viewer";\n\t"read" if "viewer";\n\n\t"create_child" if "collaborator";\n\t"modify" if "admin";\n\n\t# Permissions implied by roles on this resource's parent (Fleet). Fleet\n\t# privileges allow a user to see and potentially administer the Silo,\n\t# but they do not give anyone permission to look at anything inside the\n\t# Silo. To achieve this, we use permission rules here. (If we granted\n\t# Fleet administrators _roles_ on the Silo, then those would cascade\n\t# into the Silo as well.)\n\trelations = { parent_fleet: Fleet };\n\t"read" if "viewer" on "parent_fleet";\n\t"modify" if "collaborator" on "parent_fleet";\n\n\t# external authenticator has to create silo users\n\t"list_children" if "external-authenticator" on "parent_fleet";\n\t"create_child" if "external-authenticator" on "parent_fleet";\n}\n\nhas_relation(fleet: Fleet, "parent_fleet", silo: Silo)\n\tif silo.fleet = fleet;\n\n# As a special case, all authenticated users can read their own Silo. That's\n# not quite the same as having the "viewer" role. For example, they cannot list\n# Organizations in the Silo.\n#\n# One reason this is necessary is because if an unprivileged user tries to\n# create an Organization using "POST /organizations", they should get back a 403\n# (which implies they're able to see /organizations, which is essentially seeing\n# the Silo itself) rather than a 404. This behavior isn't a hard constraint\n# (i.e., you could reasonably get a 404 for an API you're not allowed to call).\n# Nor is the implementation (i.e., we could special-case this endpoint somehow).\n# But granting this permission is the simplest way to keep this endpoint's\n# behavior consistent with the rest of the API.\n#\n# This rule is also used to determine if a user can list the identity providers\n# in the Silo (which they should be able to), since that's predicated on being\n# able to read the Silo.\n#\n# It's unclear what else would break if users couldn't see their own Silo.\nhas_permission(actor: AuthenticatedActor, "read", silo: Silo)\n\tif silo in actor.silo;\n\nresource Project {\n\tpermissions = [\n\t "list_children",\n\t "modify",\n\t "read",\n\t "create_child",\n\t];\n\troles = [ "admin", "collaborator", "viewer" ];\n\n\t# Roles implied by other roles on this resource\n\t"viewer" if "collaborator";\n\t"collaborator" if "admin";\n\n\t# Permissions granted directly by roles on this resource\n\t"list_children" if "viewer";\n\t"read" if "viewer";\n\t"create_child" if "collaborator";\n\t"modify" if "admin";\n\n\t# Roles implied by roles on this resource's parent (Silo)\n\trelations = { parent_silo: Silo };\n\t"admin" if "collaborator" on "parent_silo";\n\t"viewer" if "viewer" on "parent_silo";\n}\nhas_relation(silo: Silo, "parent_silo", project: Project)\n\tif project.silo = silo;\n\n#\n# GENERAL RESOURCES OUTSIDE THE SILO/PROJECT HIERARCHY\n#\n# Many resources use snippets of Polar generated by the `authz_resource!` Rust\n# macro. Some resources require custom Polar code. Those appear here.\n#\n\nresource Certificate {\n\tpermissions = [ "read", "modify" ];\n\trelations = { parent_silo: Silo, parent_fleet: Fleet };\n\n\t# Fleet-level and silo-level roles both grant privileges on certificates.\n\t"read" if "admin" on "parent_silo";\n\t"modify" if "admin" on "parent_silo";\n\t"read" if "admin" on "parent_fleet";\n\t"modify" if "admin" on "parent_fleet";\n}\nhas_relation(silo: Silo, "parent_silo", certificate: Certificate)\n\tif certificate.silo = silo;\nhas_relation(fleet: Fleet, "parent_fleet", certificate: Certificate)\n\tif certificate.silo.fleet = fleet;\n\nresource SiloUser {\n\tpermissions = [\n\t "list_children",\n\t "modify",\n\t "read",\n\t "create_child",\n\t];\n\n\t# Fleet and Silo administrators can manage a Silo's users. This is one\n\t# of the only areas of Silo configuration that Fleet Administrators have\n\t# permissions on.\n\trelations = { parent_silo: Silo, parent_fleet: Fleet };\n\t"list_children" if "read" on "parent_silo";\n\t"read" if "read" on "parent_silo";\n\t"modify" if "admin" on "parent_silo";\n\t"create_child" if "admin" on "parent_silo";\n\t"list_children" if "read" on "parent_fleet";\n\t"read" if "read" on "parent_fleet";\n\t"modify" if "admin" on "parent_fleet";\n\t"create_child" if "admin" on "parent_fleet";\n}\nhas_relation(silo: Silo, "parent_silo", user: SiloUser)\n\tif user.silo = silo;\nhas_relation(fleet: Fleet, "parent_fleet", user: SiloUser)\n\tif user.silo.fleet = fleet;\n\n# authenticated actors have all permissions on themselves\nhas_permission(actor: AuthenticatedActor, _perm: String, silo_user: SiloUser)\n if actor.equals_silo_user(silo_user);\n\nhas_permission(actor: AuthenticatedActor, "read", silo_user: SiloUser)\n if silo_user.silo in actor.silo;\n\nresource SiloGroup {\n\tpermissions = [\n\t "list_children",\n\t "modify",\n\t "read",\n\t "create_child",\n\t];\n\n\trelations = { parent_silo: Silo };\n\t"list_children" if "read" on "parent_silo";\n\t"read" if "read" on "parent_silo";\n\t"modify" if "admin" on "parent_silo";\n\t"create_child" if "admin" on "parent_silo";\n}\nhas_relation(silo: Silo, "parent_silo", group: SiloGroup)\n\tif group.silo = silo;\n\nresource SshKey {\n\tpermissions = [ "read", "modify" ];\n\trelations = { silo_user: SiloUser };\n\n\t"read" if "read" on "silo_user";\n\t"modify" if "modify" on "silo_user";\n}\nhas_relation(user: SiloUser, "silo_user", ssh_key: SshKey)\n\tif ssh_key.silo_user = user;\n\nresource IdentityProvider {\n\tpermissions = [\n\t "read",\n\t "modify",\n\t "create_child",\n\t "list_children",\n\t];\n\trelations = { parent_silo: Silo, parent_fleet: Fleet };\n\n\t# Silo-level roles grant privileges on identity providers.\n\t"read" if "viewer" on "parent_silo";\n\t"list_children" if "viewer" on "parent_silo";\n\t"modify" if "admin" on "parent_silo";\n\t"create_child" if "admin" on "parent_silo";\n\n\t# Fleet-level roles also grant privileges on identity providers.\n\t"read" if "viewer" on "parent_fleet";\n\t"list_children" if "viewer" on "parent_fleet";\n\t"modify" if "admin" on "parent_fleet";\n\t"create_child" if "admin" on "parent_fleet";\n}\nhas_relation(silo: Silo, "parent_silo", identity_provider: IdentityProvider)\n\tif identity_provider.silo = silo;\nhas_relation(fleet: Fleet, "parent_fleet", collection: IdentityProvider)\n\tif collection.silo.fleet = fleet;\n\nresource SamlIdentityProvider {\n\tpermissions = [\n\t "read",\n\t "modify",\n\t "create_child",\n\t "list_children",\n\t];\n\trelations = { parent_silo: Silo, parent_fleet: Fleet };\n\n\t# Silo-level roles grant privileges on identity providers.\n\t"read" if "viewer" on "parent_silo";\n\t"list_children" if "viewer" on "parent_silo";\n\t"modify" if "admin" on "parent_silo";\n\t"create_child" if "admin" on "parent_silo";\n\n\t# Fleet-level roles also grant privileges on identity providers.\n\t"read" if "viewer" on "parent_fleet";\n\t"list_children" if "viewer" on "parent_fleet";\n\t"modify" if "admin" on "parent_fleet";\n\t"create_child" if "admin" on "parent_fleet";\n}\nhas_relation(silo: Silo, "parent_silo", saml_identity_provider: SamlIdentityProvider)\n\tif saml_identity_provider.silo = silo;\nhas_relation(fleet: Fleet, "parent_fleet", collection: SamlIdentityProvider)\n\tif collection.silo.fleet = fleet;\n\n#\n# SYNTHETIC RESOURCES OUTSIDE THE SILO HIERARCHY\n#\n# The resources here do not correspond to anything that appears explicitly in\n# the API or is stored in the database. These are used either at the top level\n# of the API path (e.g., "/v1/system/ip-pools") or as an implementation detail of the system\n# (in the case of console sessions and "Database"). The policies are\n# either statically-defined in this file or driven by role assignments on the\n# Fleet. None of these resources defines their own roles.\n#\n\n# Describes the policy for reading and modifying DNS configuration\n# (both internal and external)\nresource DnsConfig {\n\tpermissions = [ "read", "modify" ];\n\trelations = { parent_fleet: Fleet };\n\t# "external-authenticator" requires these permissions because that's the\n\t# context that Nexus uses when creating and deleting Silos. These\n\t# operations necessarily need to read and modify DNS configuration.\n\t"read" if "external-authenticator" on "parent_fleet";\n\t"modify" if "external-authenticator" on "parent_fleet";\n\t# "admin" on the parent fleet also gets these permissions, primarily for\n\t# the test suite.\n\t"read" if "admin" on "parent_fleet";\n\t"modify" if "admin" on "parent_fleet";\n}\nhas_relation(fleet: Fleet, "parent_fleet", dns_config: DnsConfig)\n\tif dns_config.fleet = fleet;\n\n# Describes the policy for accessing blueprints\nresource BlueprintConfig {\n\tpermissions = [\n\t "list_children", # list blueprints\n\t "create_child", # create blueprint\n\t "read", # read the current target\n\t "modify", # change the current target\n\t];\n\n\trelations = { parent_fleet: Fleet };\n\t"create_child" if "admin" on "parent_fleet";\n\t"modify" if "admin" on "parent_fleet";\n\t"list_children" if "viewer" on "parent_fleet";\n\t"read" if "viewer" on "parent_fleet";\n}\nhas_relation(fleet: Fleet, "parent_fleet", list: BlueprintConfig)\n\tif list.fleet = fleet;\n\n# Describes the policy for accessing blueprints\nresource TargetReleaseConfig {\n\tpermissions = [\n\t "read", # read the current target release\n\t "modify", # change the current target release\n\t];\n\n\trelations = { parent_fleet: Fleet };\n\t"read" if "viewer" on "parent_fleet";\n\t"modify" if "admin" on "parent_fleet";\n}\nhas_relation(fleet: Fleet, "parent_fleet", resource: TargetReleaseConfig)\n\tif resource.fleet = fleet;\n\n# Describes the policy for reading and modifying low-level inventory\nresource Inventory {\n\tpermissions = [ "read", "modify" ];\n\trelations = { parent_fleet: Fleet };\n\t"read" if "viewer" on "parent_fleet";\n\t"modify" if "admin" on "parent_fleet";\n}\nhas_relation(fleet: Fleet, "parent_fleet", inventory: Inventory)\n\tif inventory.fleet = fleet;\n\n# Describes the policy for accessing "/v1/system/ip-pools" in the API\nresource IpPoolList {\n\tpermissions = [\n\t "list_children",\n\t "modify",\n\t "create_child",\n\t];\n\n\t# Fleet Administrators can create or modify the IP Pools list.\n\trelations = { parent_fleet: Fleet };\n\t"modify" if "admin" on "parent_fleet";\n\t"create_child" if "admin" on "parent_fleet";\n\n\t# Fleet Viewers can list IP Pools\n\t"list_children" if "viewer" on "parent_fleet";\n}\nhas_relation(fleet: Fleet, "parent_fleet", ip_pool_list: IpPoolList)\n\tif ip_pool_list.fleet = fleet;\n\n# Any authenticated user can create a child of a provided IP Pool.\n# This is necessary to use the pools when provisioning instances.\nhas_permission(actor: AuthenticatedActor, "create_child", ip_pool: IpPool)\n\tif silo in actor.silo and silo.fleet = ip_pool.fleet;\n\n# Describes the policy for creating and managing web console sessions.\nresource ConsoleSessionList {\n\tpermissions = [ "create_child" ];\n\trelations = { parent_fleet: Fleet };\n\t"create_child" if "external-authenticator" on "parent_fleet";\n}\nhas_relation(fleet: Fleet, "parent_fleet", collection: ConsoleSessionList)\n\tif collection.fleet = fleet;\n\n# Describes the policy for creating and managing device authorization requests.\nresource DeviceAuthRequestList {\n\tpermissions = [ "create_child" ];\n\trelations = { parent_fleet: Fleet };\n\t"create_child" if "external-authenticator" on "parent_fleet";\n}\nhas_relation(fleet: Fleet, "parent_fleet", collection: DeviceAuthRequestList)\n\tif collection.fleet = fleet;\n\n# Describes the policy for creating and managing Silo certificates\nresource SiloCertificateList {\n\tpermissions = [ "list_children", "create_child" ];\n\n\trelations = { parent_silo: Silo, parent_fleet: Fleet };\n\n\t# Both Fleet and Silo administrators can see and modify the Silo's\n\t# certificates.\n\t"list_children" if "admin" on "parent_silo";\n\t"list_children" if "admin" on "parent_fleet";\n\t"create_child" if "admin" on "parent_silo";\n\t"create_child" if "admin" on "parent_fleet";\n}\nhas_relation(silo: Silo, "parent_silo", collection: SiloCertificateList)\n\tif collection.silo = silo;\nhas_relation(fleet: Fleet, "parent_fleet", collection: SiloCertificateList)\n\tif collection.silo.fleet = fleet;\n\n# Describes the policy for creating and managing Silo identity providers\nresource SiloIdentityProviderList {\n\tpermissions = [ "list_children", "create_child" ];\n\n\trelations = { parent_silo: Silo, parent_fleet: Fleet };\n\n\t# Everyone who can read the Silo (which includes all the users in the\n\t# Silo) can see the identity providers in it.\n\t"list_children" if "read" on "parent_silo";\n\n\t# Fleet and Silo administrators can manage the Silo's identity provider\n\t# configuration. This is one of the only areas of Silo configuration\n\t# that Fleet Administrators have permissions on. This is also one of\n\t# the only cases where we need to look two levels up the hierarchy to\n\t# see if somebody has the right permission. For most other things,\n\t# permissions cascade down the hierarchy so we only need to look at the\n\t# parent.\n\t"create_child" if "admin" on "parent_silo";\n\t"create_child" if "admin" on "parent_fleet";\n}\nhas_relation(silo: Silo, "parent_silo", collection: SiloIdentityProviderList)\n\tif collection.silo = silo;\nhas_relation(fleet: Fleet, "parent_fleet", collection: SiloIdentityProviderList)\n\tif collection.silo.fleet = fleet;\n\n# Describes the policy for creating and managing Silo users (mostly intended for\n# API-managed users)\nresource SiloUserList {\n\tpermissions = [ "list_children", "create_child" ];\n\n\trelations = { parent_silo: Silo, parent_fleet: Fleet };\n\n\t# Everyone who can read the Silo (which includes all the users in the\n\t# Silo) can see the users in it.\n\t"list_children" if "read" on "parent_silo";\n\n\t# Fleet and Silo administrators can manage the Silo's users. This is\n\t# one of the only areas of Silo configuration that Fleet Administrators\n\t# have permissions on. This is also one of the few cases (so far) where\n\t# we need to look two levels up the hierarchy to see if somebody has the\n\t# right permission. For most other things, permissions cascade down the\n\t# hierarchy so we only need to look at the parent.\n\t"create_child" if "admin" on "parent_silo";\n\t"list_children" if "admin" on "parent_fleet";\n\t"create_child" if "admin" on "parent_fleet";\n}\nhas_relation(silo: Silo, "parent_silo", collection: SiloUserList)\n\tif collection.silo = silo;\nhas_relation(fleet: Fleet, "parent_fleet", collection: SiloUserList)\n\tif collection.silo.fleet = fleet;\n\n# These rules grants the external authenticator role the permissions it needs to\n# read silo users and modify their sessions. This is necessary for login to\n# work.\nhas_permission(actor: AuthenticatedActor, "read", silo: Silo)\n\tif has_role(actor, "external-authenticator", silo.fleet);\nhas_permission(actor: AuthenticatedActor, "read", user: SiloUser)\n\tif has_role(actor, "external-authenticator", user.silo.fleet);\nhas_permission(actor: AuthenticatedActor, "modify", user: SiloUser)\n\tif has_role(actor, "external-authenticator", user.silo.fleet);\nhas_permission(actor: AuthenticatedActor, "read", group: SiloGroup)\n\tif has_role(actor, "external-authenticator", group.silo.fleet);\nhas_permission(actor: AuthenticatedActor, "modify", group: SiloGroup)\n\tif has_role(actor, "external-authenticator", group.silo.fleet);\n\nhas_permission(actor: AuthenticatedActor, "read", session: ConsoleSession)\n\tif has_role(actor, "external-authenticator", session.fleet);\nhas_permission(actor: AuthenticatedActor, "modify", session: ConsoleSession)\n\tif has_role(actor, "external-authenticator", session.fleet);\n\n# All authenticated users can read and delete device authn requests because\n# by necessity these operations happen before we've figured out what user (or\n# even Silo) the device auth is associated with. Any user can claim a device\n# auth request with the right user code (that's how it works) -- it's the user\n# code and associated logic that prevents unauthorized access here.\nhas_permission(_actor: AuthenticatedActor, "read", _device_auth: DeviceAuthRequest);\nhas_permission(_actor: AuthenticatedActor, "modify", _device_auth: DeviceAuthRequest);\n\nhas_permission(actor: AuthenticatedActor, "read", device_token: DeviceAccessToken)\n\tif has_role(actor, "external-authenticator", device_token.fleet);\n\nhas_permission(actor: AuthenticatedActor, "read", identity_provider: IdentityProvider)\n\tif has_role(actor, "external-authenticator", identity_provider.silo.fleet);\n\nhas_permission(actor: AuthenticatedActor, "read", saml_identity_provider: SamlIdentityProvider)\n\tif has_role(actor, "external-authenticator", saml_identity_provider.silo.fleet);\n\n# Describes the policy for who can access the internal database.\nresource Database {\n\tpermissions = [\n\t # "query" is required to perform any query against the database,\n\t # whether a read or write query. This is checked when an operation\n\t # checks out a database connection from the connection pool.\n\t #\n\t # Any authenticated user gets this permission. There's generally\n\t # some other authz check involved in the database query. For\n\t # example, if you're querying the database to "read" a "Project", we\n\t # should also be checking that. So why do we do this at all? It's\n\t # a belt-and-suspenders measure so that if we somehow introduced an\n\t # unauthenticated code path that hits the database, it cannot be\n\t # used to DoS the database because we won't allow the operation to\n\t # make the query. (As long as the code path _is_ authenticated, we\n\t # can use throttling mechanisms to prevent DoS.)\n\t "query",\n\n\t # "modify" is required to populate database data that's delivered\n\t # with the system. It should also be required for schema changes,\n\t # when we support those. This is separate from "query" so that we\n\t # cannot accidentally invoke these code paths from API calls and\n\t # other general functions.\n\t "modify"\n\t];\n}\n\n# All authenticated users have the "query" permission on the database.\nhas_permission(_actor: AuthenticatedActor, "query", _resource: Database);\n\n# The "db-init" user is the only one with the "modify" permission.\nhas_permission(USER_DB_INIT: AuthenticatedActor, "modify", _resource: Database);\nhas_permission(USER_DB_INIT: AuthenticatedActor, "create_child", _resource: IpPoolList);\n# It also has "admin" on the internal silo to populate it with built-in resources.\n# TODO-completeness: actually limit to just internal silo and not all silos\nhas_role(USER_DB_INIT: AuthenticatedActor, "admin", _silo: Silo);\n\n# Allow the internal API admin permissions on all silos.\nhas_role(USER_INTERNAL_API: AuthenticatedActor, "admin", _silo: Silo);\n\nresource WebhookSecret {\n\tpermissions = [ "read", "modify" ];\n\trelations = { parent_alert_receiver: AlertReceiver };\n\n\t"read" if "read" on "parent_alert_receiver";\n\t"modify" if "modify" on "parent_alert_receiver";\n}\n\nhas_relation(rx: AlertReceiver, "parent_alert_receiver", secret: WebhookSecret)\n\tif secret.alert_receiver = rx;\n\nresource AlertClassList {\n\tpermissions = [ "list_children" ];\n\trelations = { parent_fleet: Fleet };\n\n\t"list_children" if "viewer" on "parent_fleet";\n}\n\nhas_relation(fleet: Fleet, "parent_fleet", collection: AlertClassList)\n\tif collection.fleet = fleet;\n\n\n\n resource Disk {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_project: Project };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(parent: Project, "containing_project", child: Disk)\n if child.project = parent;\n \n\n resource Snapshot {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_project: Project };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(parent: Project, "containing_project", child: Snapshot)\n if child.project = parent;\n \n\n resource ProjectImage {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_project: Project };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(parent: Project, "containing_project", child: ProjectImage)\n if child.project = parent;\n \n\n resource AffinityGroup {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_project: Project };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(parent: Project, "containing_project", child: AffinityGroup)\n if child.project = parent;\n \n\n resource AntiAffinityGroup {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_project: Project };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(parent: Project, "containing_project", child: AntiAffinityGroup)\n if child.project = parent;\n \n\n resource Instance {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_project: Project };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(parent: Project, "containing_project", child: Instance)\n if child.project = parent;\n \n\n resource IpPool {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: IpPool)\n if child.fleet = fleet;\n \n\n resource InstanceNetworkInterface {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = {\n containing_project: Project,\n parent: Instance\n };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(project: Project, "containing_project", child: InstanceNetworkInterface)\n if has_relation(project, "containing_project", child.instance);\n\n has_relation(parent: Instance, "parent", child: InstanceNetworkInterface)\n if child.instance = parent;\n \n\n resource Vpc {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_project: Project };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(parent: Project, "containing_project", child: Vpc)\n if child.project = parent;\n \n\n resource VpcRouter {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = {\n containing_project: Project,\n parent: Vpc\n };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(project: Project, "containing_project", child: VpcRouter)\n if has_relation(project, "containing_project", child.vpc);\n\n has_relation(parent: Vpc, "parent", child: VpcRouter)\n if child.vpc = parent;\n \n\n resource InternetGateway {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = {\n containing_project: Project,\n parent: Vpc\n };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(project: Project, "containing_project", child: InternetGateway)\n if has_relation(project, "containing_project", child.vpc);\n\n has_relation(parent: Vpc, "parent", child: InternetGateway)\n if child.vpc = parent;\n \n\n resource InternetGatewayIpPool {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = {\n containing_project: Project,\n parent: InternetGateway\n };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(project: Project, "containing_project", child: InternetGatewayIpPool)\n if has_relation(project, "containing_project", child.internet_gateway);\n\n has_relation(parent: InternetGateway, "parent", child: InternetGatewayIpPool)\n if child.internet_gateway = parent;\n \n\n resource InternetGatewayIpAddress {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = {\n containing_project: Project,\n parent: InternetGateway\n };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(project: Project, "containing_project", child: InternetGatewayIpAddress)\n if has_relation(project, "containing_project", child.internet_gateway);\n\n has_relation(parent: InternetGateway, "parent", child: InternetGatewayIpAddress)\n if child.internet_gateway = parent;\n \n\n resource RouterRoute {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = {\n containing_project: Project,\n parent: VpcRouter\n };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(project: Project, "containing_project", child: RouterRoute)\n if has_relation(project, "containing_project", child.vpc_router);\n\n has_relation(parent: VpcRouter, "parent", child: RouterRoute)\n if child.vpc_router = parent;\n \n\n resource VpcSubnet {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = {\n containing_project: Project,\n parent: Vpc\n };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(project: Project, "containing_project", child: VpcSubnet)\n if has_relation(project, "containing_project", child.vpc);\n\n has_relation(parent: Vpc, "parent", child: VpcSubnet)\n if child.vpc = parent;\n \n\n resource FloatingIp {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_project: Project };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(parent: Project, "containing_project", child: FloatingIp)\n if child.project = parent;\n \n\n resource Image {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_silo: Silo };\n "list_children" if "viewer" on "containing_silo";\n "read" if "viewer" on "containing_silo";\n "modify" if "collaborator" on "containing_silo";\n "create_child" if "collaborator" on "containing_silo";\n }\n\n has_relation(parent: Silo, "containing_silo", child: Image)\n if child.silo = parent;\n \n\n resource SiloImage {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_silo: Silo };\n "list_children" if "viewer" on "containing_silo";\n "read" if "viewer" on "containing_silo";\n "modify" if "collaborator" on "containing_silo";\n "create_child" if "collaborator" on "containing_silo";\n }\n\n has_relation(parent: Silo, "containing_silo", child: SiloImage)\n if child.silo = parent;\n \n\n resource AddressLot {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: AddressLot)\n if child.fleet = fleet;\n \n\n resource Blueprint {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: Blueprint)\n if child.fleet = fleet;\n \n\n resource LoopbackAddress {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: LoopbackAddress)\n if child.fleet = fleet;\n \n\n\n resource ConsoleSession {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: ConsoleSession)\n if child.fleet = fleet;\n \n\n resource DeviceAuthRequest {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: DeviceAuthRequest)\n if child.fleet = fleet;\n \n\n resource DeviceAccessToken {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: DeviceAccessToken)\n if child.fleet = fleet;\n \n\n resource PhysicalDisk {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: PhysicalDisk)\n if child.fleet = fleet;\n \n\n resource Rack {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: Rack)\n if child.fleet = fleet;\n \n\n resource RoleBuiltin {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: RoleBuiltin)\n if child.fleet = fleet;\n \n\n\n\n\n\n resource SupportBundle {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: SupportBundle)\n if child.fleet = fleet;\n \n\n\n\n resource Sled {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: Sled)\n if child.fleet = fleet;\n \n\n resource TufRepo {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: TufRepo)\n if child.fleet = fleet;\n \n\n resource TufArtifact {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: TufArtifact)\n if child.fleet = fleet;\n \n\n resource Alert {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: Alert)\n if child.fleet = fleet;\n \n\n resource AlertReceiver {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: AlertReceiver)\n if child.fleet = fleet;\n \n\n\n resource Zpool {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: Zpool)\n if child.fleet = fleet;\n \n\n resource Service {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: Service)\n if child.fleet = fleet;\n \n\n resource UserBuiltin {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: UserBuiltin)\n if child.fleet = fleet;\n file = nexus/auth/src/authz/oso_generic.rs:89
712025-06-03T00:51:06.591ZINFOnexus (ServerContext): Setting up resolver using DNS servers for subnet: Ipv6Subnet { net: Ipv6Net { addr: fd00:1122:3344::, width: 48 } } file = nexus/src/context.rs:222
722025-06-03T00:51:06.593ZINFOnexus (DnsResolver): new DNS resolver addresses = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = internal-dns/resolver/src/resolver.rs:111
732025-06-03T00:51:06.597ZINFOnexus (ServerContext): Setting up qorb database pool from DNS dns_addrs = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = nexus/src/context.rs:278
742025-06-03T00:51:06.603ZDEBGnexus (ServerContext): registered USDT probes
752025-06-03T00:51:06.779ZINFOnexus: Database schema version is up to date desired_version = 144.0.0 file = nexus/db-queries/src/db/datastore/db_metadata.rs:143 found_version = 144.0.0
762025-06-03T00:51:06.783ZINFOnexus (SEC): SEC running file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/steno-0.4.1/src/sec.rs:813 sec_id = a2861392-5fd2-462b-84e5-6f36c716263a
772025-06-03T00:51:06.840ZDEBGnexus (clickhouse-client): registered USDT probes id = e4b0b6b3-3d80-4de6-bc4d-168da7795df5
782025-06-03T00:51:06.851ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
792025-06-03T00:51:06.862ZDEBGnexus (DataLoader): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
802025-06-03T00:51:06.865ZDEBGnexus (DataLoader): attempting to create built-in users
812025-06-03T00:51:06.865ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
822025-06-03T00:51:06.865ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
832025-06-03T00:51:06.887ZINFOnexus (MgsUpdateDriver): starting MgsUpdateDriver file = nexus/mgs-updates/src/driver.rs:148
842025-06-03T00:51:06.916ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::a]:12221 method = GET path = /bgtasks versions = all
852025-06-03T00:51:06.916ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::a]:12221 method = POST path = /bgtasks/activate versions = all
862025-06-03T00:51:06.916ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::a]:12221 method = GET path = /bgtasks/view/{bgtask_name} versions = all
872025-06-03T00:51:06.916ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::a]:12221 method = GET path = /clickhouse/policy versions = all
882025-06-03T00:51:06.916ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::a]:12221 method = POST path = /clickhouse/policy versions = all
892025-06-03T00:51:06.916ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::a]:12221 method = POST path = /crucible/0/upstairs/{upstairs_id}/downstairs/{downstairs_id}/stop-request versions = all
902025-06-03T00:51:06.916ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::a]:12221 method = POST path = /crucible/0/upstairs/{upstairs_id}/downstairs/{downstairs_id}/stopped versions = all
912025-06-03T00:51:06.916ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::a]:12221 method = POST path = /crucible/0/upstairs/{upstairs_id}/repair/{repair_id}/progress versions = all
922025-06-03T00:51:06.916ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::a]:12221 method = POST path = /crucible/0/upstairs/{upstairs_id}/repair-finish versions = all
932025-06-03T00:51:06.916ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::a]:12221 method = POST path = /crucible/0/upstairs/{upstairs_id}/repair-start versions = all
942025-06-03T00:51:06.916ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::a]:12221 method = POST path = /demo-saga versions = all
952025-06-03T00:51:06.917ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::a]:12221 method = POST path = /demo-saga/{demo_saga_id}/complete versions = all
962025-06-03T00:51:06.917ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::a]:12221 method = GET path = /deployment/blueprints/all versions = all
972025-06-03T00:51:06.917ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::a]:12221 method = DELETE path = /deployment/blueprints/all/{blueprint_id} versions = all
982025-06-03T00:51:06.917ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::a]:12221 method = GET path = /deployment/blueprints/all/{blueprint_id} versions = all
992025-06-03T00:51:06.917ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::a]:12221 method = POST path = /deployment/blueprints/import versions = all
1002025-06-03T00:51:06.917ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::a]:12221 method = POST path = /deployment/blueprints/regenerate versions = all
1012025-06-03T00:51:06.917ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::a]:12221 method = GET path = /deployment/blueprints/target versions = all
1022025-06-03T00:51:06.917ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::a]:12221 method = POST path = /deployment/blueprints/target versions = all
1032025-06-03T00:51:06.917ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::a]:12221 method = PUT path = /deployment/blueprints/target/enabled versions = all
1042025-06-03T00:51:06.917ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::a]:12221 method = POST path = /disk/{disk_id}/remove-read-only-parent versions = all
1052025-06-03T00:51:06.917ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::a]:12221 method = PUT path = /disks/{disk_id} versions = all
1062025-06-03T00:51:06.917ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::a]:12221 method = GET path = /experimental/v1/system/support-bundles versions = all
1072025-06-03T00:51:06.917ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::a]:12221 method = POST path = /experimental/v1/system/support-bundles versions = all
1082025-06-03T00:51:06.917ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::a]:12221 method = DELETE path = /experimental/v1/system/support-bundles/{bundle_id} versions = all
1092025-06-03T00:51:06.917ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::a]:12221 method = GET path = /experimental/v1/system/support-bundles/{bundle_id} versions = all
1102025-06-03T00:51:06.917ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::a]:12221 method = GET path = /experimental/v1/system/support-bundles/{bundle_id}/download versions = all
1112025-06-03T00:51:06.917ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::a]:12221 method = HEAD path = /experimental/v1/system/support-bundles/{bundle_id}/download versions = all
1122025-06-03T00:51:06.917ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::a]:12221 method = GET path = /experimental/v1/system/support-bundles/{bundle_id}/download/{file} versions = all
1132025-06-03T00:51:06.917ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::a]:12221 method = HEAD path = /experimental/v1/system/support-bundles/{bundle_id}/download/{file} versions = all
1142025-06-03T00:51:06.917ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::a]:12221 method = GET path = /experimental/v1/system/support-bundles/{bundle_id}/index versions = all
1152025-06-03T00:51:06.917ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::a]:12221 method = POST path = /instances/{instance_id}/migrate versions = all
1162025-06-03T00:51:06.917ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::a]:12221 method = POST path = /metrics/collectors versions = all
1172025-06-03T00:51:06.917ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::a]:12221 method = GET path = /metrics/collectors/{collector_id}/producers versions = all
1182025-06-03T00:51:06.917ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::a]:12221 method = POST path = /metrics/producers versions = all
1192025-06-03T00:51:06.917ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::a]:12221 method = GET path = /mgs-updates versions = all
1202025-06-03T00:51:06.917ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::a]:12221 method = GET path = /nat/ipv4/changeset/{from_gen} versions = all
1212025-06-03T00:51:06.917ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::a]:12221 method = GET path = /oximeter/read-policy versions = all
1222025-06-03T00:51:06.917ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::a]:12221 method = POST path = /oximeter/read-policy versions = all
1232025-06-03T00:51:06.917ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::a]:12221 method = POST path = /physical-disk/expunge versions = all
1242025-06-03T00:51:06.917ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::a]:12221 method = GET path = /probes/{sled} versions = all
1252025-06-03T00:51:06.917ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::a]:12221 method = PUT path = /racks/{rack_id}/initialization-complete versions = all
1262025-06-03T00:51:06.917ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::a]:12221 method = GET path = /sagas versions = all
1272025-06-03T00:51:06.917ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::a]:12221 method = GET path = /sagas/{saga_id} versions = all
1282025-06-03T00:51:06.917ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::a]:12221 method = GET path = /sled-agents/{sled_id} versions = all
1292025-06-03T00:51:06.917ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::a]:12221 method = POST path = /sled-agents/{sled_id} versions = all
1302025-06-03T00:51:06.917ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::a]:12221 method = POST path = /sled-agents/{sled_id}/firewall-rules-update versions = all
1312025-06-03T00:51:06.917ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::a]:12221 method = POST path = /sleds/add versions = all
1322025-06-03T00:51:06.917ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::a]:12221 method = POST path = /sleds/expunge versions = all
1332025-06-03T00:51:06.917ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::a]:12221 method = GET path = /sleds/uninitialized versions = all
1342025-06-03T00:51:06.917ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::a]:12221 method = PUT path = /switch/{switch_id} versions = all
1352025-06-03T00:51:06.917ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::a]:12221 method = GET path = /v1/ping versions = all
1362025-06-03T00:51:06.917ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::a]:12221 method = PUT path = /vmms/{propolis_id} versions = all
1372025-06-03T00:51:06.917ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::a]:12221 method = POST path = /volume/{volume_id}/remove-read-only-parent versions = all
1382025-06-03T00:51:06.917ZINFOnexus (dropshot_internal): listening file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:328 local_addr = [fd00:1122:3344:101::a]:12221
1392025-06-03T00:51:06.917ZDEBGnexus (dropshot_internal): successfully registered DTrace USDT probes local_addr = [fd00:1122:3344:101::a]:12221
1402025-06-03T00:51:06.917ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
1412025-06-03T00:51:06.917ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
1422025-06-03T00:51:06.938ZWARNnexus: Cannot look up rack: Object (of type ById(d6831ba0-55f0-4191-8dc2-ea15e55ab0ae)) not found: rack file = nexus/src/app/rack.rs:781
1432025-06-03T00:51:07.018ZINFOnexus (DataLoader): created 0 built-in users file = nexus/db-queries/src/db/datastore/silo_user.rs:393
1442025-06-03T00:51:07.018ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
1452025-06-03T00:51:07.021ZDEBGnexus (DataLoader): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1462025-06-03T00:51:07.023ZDEBGnexus (DataLoader): attempting to create built-in roles
1472025-06-03T00:51:07.023ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
1482025-06-03T00:51:07.023ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1492025-06-03T00:51:07.119ZINFOnexus (DataLoader): created 0 built-in roles file = nexus/db-queries/src/db/datastore/role.rs:84
1502025-06-03T00:51:07.119ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
1512025-06-03T00:51:07.121ZDEBGnexus (DataLoader): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1522025-06-03T00:51:07.121ZDEBGnexus (DataLoader): attempting to create built-in role assignments
1532025-06-03T00:51:07.121ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
1542025-06-03T00:51:07.121ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1552025-06-03T00:51:07.576ZINFOnexus (DataLoader): created 0 built-in role assignments file = nexus/db-queries/src/db/datastore/role.rs:113
1562025-06-03T00:51:07.576ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
1572025-06-03T00:51:07.577ZDEBGnexus (DataLoader): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1582025-06-03T00:51:07.577ZDEBGnexus (DataLoader): attempting to create built-in silos
1592025-06-03T00:51:07.577ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
1602025-06-03T00:51:07.578ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1612025-06-03T00:51:07.774ZINFOnexus (DataLoader): created 2 built-in silos file = nexus/db-queries/src/db/datastore/silo.rs:93
1622025-06-03T00:51:07.774ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
1632025-06-03T00:51:07.775ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1642025-06-03T00:51:07.797ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
1652025-06-03T00:51:07.798ZDEBGnexus (DataLoader): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1662025-06-03T00:51:07.798ZDEBGnexus (DataLoader): attempting to create built-in projects
1672025-06-03T00:51:07.798ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
1682025-06-03T00:51:07.799ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1692025-06-03T00:51:07.869ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
1702025-06-03T00:51:07.869ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1712025-06-03T00:51:08.431ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
1722025-06-03T00:51:08.431ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1732025-06-03T00:51:08.940ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
1742025-06-03T00:51:08.940ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
1752025-06-03T00:51:08.941ZWARNnexus: Cannot look up rack: Object (of type ById(d6831ba0-55f0-4191-8dc2-ea15e55ab0ae)) not found: rack file = nexus/src/app/rack.rs:781
1762025-06-03T00:51:09.220ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
1772025-06-03T00:51:09.223ZDEBGnexus (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) } result = Ok(())
1782025-06-03T00:51:09.223ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
1792025-06-03T00:51:09.224ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1802025-06-03T00:51:09.274ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
1812025-06-03T00:51:09.275ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1822025-06-03T00:51:09.336ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
1832025-06-03T00:51:09.338ZDEBGnexus (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) } result = Ok(())
1842025-06-03T00:51:09.338ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
1852025-06-03T00:51:09.338ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1862025-06-03T00:51:09.403ZINFOnexus (DataLoader): created built-in services project file = nexus/db-queries/src/db/datastore/project.rs:122
1872025-06-03T00:51:09.403ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
1882025-06-03T00:51:09.404ZDEBGnexus (DataLoader): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1892025-06-03T00:51:09.404ZDEBGnexus (DataLoader): attempting to create built-in VPCs
1902025-06-03T00:51:09.404ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
1912025-06-03T00:51:09.404ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1922025-06-03T00:51:09.417ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
1932025-06-03T00:51:09.417ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1942025-06-03T00:51:09.432ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
1952025-06-03T00:51:09.432ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1962025-06-03T00:51:09.434ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
1972025-06-03T00:51:09.434ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1982025-06-03T00:51:09.436ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
1992025-06-03T00:51:09.436ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2002025-06-03T00:51:09.439ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2012025-06-03T00:51:09.441ZDEBGnexus (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) } result = Ok(())
2022025-06-03T00:51:09.441ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2032025-06-03T00:51:09.442ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2042025-06-03T00:51:09.445ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2052025-06-03T00:51:09.446ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2062025-06-03T00:51:09.490ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2072025-06-03T00:51:09.490ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2082025-06-03T00:51:09.492ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2092025-06-03T00:51:09.495ZDEBGnexus (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) } result = Ok(())
2102025-06-03T00:51:09.495ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2112025-06-03T00:51:09.495ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2122025-06-03T00:51:09.660ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2132025-06-03T00:51:09.661ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2142025-06-03T00:51:09.662ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2152025-06-03T00:51:09.662ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2162025-06-03T00:51:09.664ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2172025-06-03T00:51:09.664ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2182025-06-03T00:51:09.665ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2192025-06-03T00:51:09.668ZDEBGnexus (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") } result = Ok(())
2202025-06-03T00:51:09.668ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2212025-06-03T00:51:09.668ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2222025-06-03T00:51:09.679ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2232025-06-03T00:51:09.679ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2242025-06-03T00:51:09.688ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2252025-06-03T00:51:09.688ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2262025-06-03T00:51:09.701ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2272025-06-03T00:51:09.701ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2282025-06-03T00:51:09.703ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2292025-06-03T00:51:09.703ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2302025-06-03T00:51:09.705ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2312025-06-03T00:51:09.705ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2322025-06-03T00:51:09.706ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2332025-06-03T00:51:09.706ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2342025-06-03T00:51:09.708ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2352025-06-03T00:51:09.708ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2362025-06-03T00:51:09.709ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2372025-06-03T00:51:09.714ZDEBGnexus (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = VpcRouter { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000001, lookup_type: ById(001de000-074c-4000-8000-000000000001) } result = Ok(())
2382025-06-03T00:51:09.714ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2392025-06-03T00:51:09.714ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2402025-06-03T00:51:09.716ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2412025-06-03T00:51:09.716ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2422025-06-03T00:51:09.717ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2432025-06-03T00:51:09.717ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2442025-06-03T00:51:09.718ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2452025-06-03T00:51:09.722ZDEBGnexus (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = VpcRouter { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000001, lookup_type: ById(001de000-074c-4000-8000-000000000001) } result = Ok(())
2462025-06-03T00:51:09.722ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2472025-06-03T00:51:09.722ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2482025-06-03T00:51:09.808ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2492025-06-03T00:51:09.808ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2502025-06-03T00:51:09.814ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2512025-06-03T00:51:09.814ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2522025-06-03T00:51:09.815ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2532025-06-03T00:51:09.816ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2542025-06-03T00:51:09.817ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2552025-06-03T00:51:09.820ZDEBGnexus (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = VpcRouter { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000001, lookup_type: ById(001de000-074c-4000-8000-000000000001) } result = Ok(())
2562025-06-03T00:51:09.820ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2572025-06-03T00:51:09.820ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2582025-06-03T00:51:09.871ZDEBGnexus (DataLoader): attempting to create built-in VPC firewall rules
2592025-06-03T00:51:09.871ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2602025-06-03T00:51:09.872ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2612025-06-03T00:51:09.874ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2622025-06-03T00:51:09.874ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2632025-06-03T00:51:09.876ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2642025-06-03T00:51:09.876ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2652025-06-03T00:51:09.889ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2662025-06-03T00:51:09.889ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2672025-06-03T00:51:09.890ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2682025-06-03T00:51:09.891ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2692025-06-03T00:51:09.892ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2702025-06-03T00:51:09.892ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2712025-06-03T00:51:09.893ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2722025-06-03T00:51:09.897ZDEBGnexus (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
2732025-06-03T00:51:09.897ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2742025-06-03T00:51:09.897ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2752025-06-03T00:51:09.898ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2762025-06-03T00:51:09.898ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2772025-06-03T00:51:09.900ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2782025-06-03T00:51:09.900ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2792025-06-03T00:51:09.901ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2802025-06-03T00:51:09.904ZDEBGnexus (DataLoader): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
2812025-06-03T00:51:09.904ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2822025-06-03T00:51:09.904ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2832025-06-03T00:51:09.921ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2842025-06-03T00:51:09.921ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2852025-06-03T00:51:09.922ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2862025-06-03T00:51:09.922ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2872025-06-03T00:51:09.923ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2882025-06-03T00:51:09.924ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2892025-06-03T00:51:09.925ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2902025-06-03T00:51:09.928ZDEBGnexus (DataLoader): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
2912025-06-03T00:51:09.928ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2922025-06-03T00:51:09.928ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2932025-06-03T00:51:10.009ZINFOnexus (DataLoader): created built-in VPC firewall rules file = nexus/db-queries/src/db/datastore/vpc.rs:285
2942025-06-03T00:51:10.009ZDEBGnexus (DataLoader): attempting to create built-in VPC Subnets
2952025-06-03T00:51:10.009ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2962025-06-03T00:51:10.009ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2972025-06-03T00:51:10.019ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2982025-06-03T00:51:10.019ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2992025-06-03T00:51:10.021ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3002025-06-03T00:51:10.022ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3012025-06-03T00:51:10.024ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3022025-06-03T00:51:10.024ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3032025-06-03T00:51:10.026ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3042025-06-03T00:51:10.026ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3052025-06-03T00:51:10.027ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3062025-06-03T00:51:10.027ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3072025-06-03T00:51:10.029ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3082025-06-03T00:51:10.032ZDEBGnexus (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
3092025-06-03T00:51:10.032ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3102025-06-03T00:51:10.032ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3112025-06-03T00:51:10.064ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3122025-06-03T00:51:10.064ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3132025-06-03T00:51:10.075ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3142025-06-03T00:51:10.076ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3152025-06-03T00:51:10.078ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3162025-06-03T00:51:10.078ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3172025-06-03T00:51:10.079ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3182025-06-03T00:51:10.082ZDEBGnexus (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
3192025-06-03T00:51:10.116ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3202025-06-03T00:51:10.116ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3212025-06-03T00:51:10.118ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3222025-06-03T00:51:10.118ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3232025-06-03T00:51:10.119ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3242025-06-03T00:51:10.120ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3252025-06-03T00:51:10.121ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3262025-06-03T00:51:10.125ZDEBGnexus (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = VpcRouter { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000001, lookup_type: ById(001de000-074c-4000-8000-000000000001) } result = Ok(())
3272025-06-03T00:51:10.125ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3282025-06-03T00:51:10.125ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3292025-06-03T00:51:10.179ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3302025-06-03T00:51:10.179ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3312025-06-03T00:51:10.181ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3322025-06-03T00:51:10.181ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3332025-06-03T00:51:10.183ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3342025-06-03T00:51:10.183ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3352025-06-03T00:51:10.185ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3362025-06-03T00:51:10.185ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3372025-06-03T00:51:10.186ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3382025-06-03T00:51:10.187ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3392025-06-03T00:51:10.188ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3402025-06-03T00:51:10.188ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3412025-06-03T00:51:10.194ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3422025-06-03T00:51:10.194ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3432025-06-03T00:51:10.195ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3442025-06-03T00:51:10.199ZDEBGnexus (DataLoader): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ById(001de000-c470-4000-8000-000000000002) } result = Ok(())
3452025-06-03T00:51:10.199ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3462025-06-03T00:51:10.200ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3472025-06-03T00:51:10.201ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3482025-06-03T00:51:10.201ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3492025-06-03T00:51:10.203ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3502025-06-03T00:51:10.203ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3512025-06-03T00:51:10.204ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3522025-06-03T00:51:10.204ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3532025-06-03T00:51:10.206ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3542025-06-03T00:51:10.209ZDEBGnexus (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
3552025-06-03T00:51:10.230ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3562025-06-03T00:51:10.231ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3572025-06-03T00:51:10.232ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3582025-06-03T00:51:10.232ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3592025-06-03T00:51:10.233ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3602025-06-03T00:51:10.233ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3612025-06-03T00:51:10.234ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3622025-06-03T00:51:10.238ZDEBGnexus (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = VpcRouter { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000001, lookup_type: ById(001de000-074c-4000-8000-000000000001) } result = Ok(())
3632025-06-03T00:51:10.238ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3642025-06-03T00:51:10.238ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3652025-06-03T00:51:10.244ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3662025-06-03T00:51:10.245ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3672025-06-03T00:51:10.250ZINFOnexus (DataLoader): created built-in services vpc subnets file = nexus/db-queries/src/db/datastore/vpc.rs:351
3682025-06-03T00:51:10.250ZINFOnexus (DataLoader): created built-in services vpc file = nexus/db-queries/src/db/datastore/vpc.rs:227
3692025-06-03T00:51:10.250ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3702025-06-03T00:51:10.250ZDEBGnexus (DataLoader): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3712025-06-03T00:51:10.250ZDEBGnexus (DataLoader): attempting to create silo users
3722025-06-03T00:51:10.250ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3732025-06-03T00:51:10.251ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3742025-06-03T00:51:10.271ZINFOnexus (DataLoader): created 2 silo users file = nexus/db-queries/src/db/datastore/silo_user.rs:418
3752025-06-03T00:51:10.271ZDEBGnexus (DataLoader): attempting to create silo user role assignments
3762025-06-03T00:51:10.271ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3772025-06-03T00:51:10.271ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3782025-06-03T00:51:10.276ZINFOnexus (DataLoader): created 2 silo user role assignments file = nexus/db-queries/src/db/datastore/silo_user.rs:445
3792025-06-03T00:51:10.276ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3802025-06-03T00:51:10.276ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3812025-06-03T00:51:10.288ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3822025-06-03T00:51:10.289ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3832025-06-03T00:51:10.300ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3842025-06-03T00:51:10.301ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3852025-06-03T00:51:10.302ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3862025-06-03T00:51:10.303ZDEBGnexus (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = IpPoolList result = Ok(())
3872025-06-03T00:51:10.303ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3882025-06-03T00:51:10.303ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3892025-06-03T00:51:10.316ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3902025-06-03T00:51:10.317ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3912025-06-03T00:51:10.318ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3922025-06-03T00:51:10.319ZDEBGnexus (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = IpPoolList result = Ok(())
3932025-06-03T00:51:10.319ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3942025-06-03T00:51:10.319ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3952025-06-03T00:51:10.470ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3962025-06-03T00:51:10.470ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3972025-06-03T00:51:10.502ZINFOnexus: populate complete file = nexus/src/app/mod.rs:523
3982025-06-03T00:51:10.502ZINFOnexus: activating background tasks file = nexus/src/app/mod.rs:532
3992025-06-03T00:51:10.505ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 1 reason = Timeout
4002025-06-03T00:51:10.505ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
4012025-06-03T00:51:10.505ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
4022025-06-03T00:51:10.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
4032025-06-03T00:51:10.506ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 1 reason = Timeout
4042025-06-03T00:51:10.506ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 1 reason = Timeout
4052025-06-03T00:51:10.506ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:53
4062025-06-03T00:51:10.506ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 1 reason = Timeout
4072025-06-03T00:51:10.506ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
4082025-06-03T00:51:10.506ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
4092025-06-03T00:51:10.506ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 1 reason = Timeout
4102025-06-03T00:51:10.506ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 1 reason = Timeout
4112025-06-03T00:51:10.506ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 1 reason = Timeout
4122025-06-03T00:51:10.506ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 1 reason = Timeout
4132025-06-03T00:51:10.506ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 1 reason = Timeout
4142025-06-03T00:51:10.506ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 1 reason = Timeout
4152025-06-03T00:51:10.506ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 1 reason = Timeout
4162025-06-03T00:51:10.506ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 1 reason = Timeout
4172025-06-03T00:51:10.506ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
4182025-06-03T00:51:10.506ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 1 reason = Timeout
4192025-06-03T00:51:10.506ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-06-03 00:41:10.506877200 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
4202025-06-03T00:51:10.506ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 1 reason = Timeout
4212025-06-03T00:51:10.506ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
4222025-06-03T00:51:10.506ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 1 reason = Timeout
4232025-06-03T00:51:10.506ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 1 reason = Timeout
4242025-06-03T00:51:10.506ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
4252025-06-03T00:51:10.507ZDEBGnexus (BackgroundTasks): activating background_task = service_firewall_rule_propagation iteration = 1 reason = Timeout
4262025-06-03T00:51:10.507ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
4272025-06-03T00:51:10.507ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
4282025-06-03T00:51:10.507ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 1 reason = Timeout
4292025-06-03T00:51:10.507ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 1 reason = Timeout
4302025-06-03T00:51:10.507ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 1 reason = Timeout
4312025-06-03T00:51:10.507ZWARNnexus (BackgroundTasks): DNS propagation: skipped background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:61 reason = no config nor servers
4322025-06-03T00:51:10.507ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 1 reason = Timeout
4332025-06-03T00:51:10.507ZDEBGnexus (BackgroundTasks): activating background_task = inventory_collection iteration = 1 reason = Timeout
4342025-06-03T00:51:10.507ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
4352025-06-03T00:51:10.507ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 1 reason = Timeout
4362025-06-03T00:51:10.507ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 1 reason = Timeout
4372025-06-03T00:51:10.507ZWARNnexus (BackgroundTasks): DNS propagation: skipped background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:61 reason = no config nor servers
4382025-06-03T00:51:10.507ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
4392025-06-03T00:51:10.507ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
4402025-06-03T00:51:10.507ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 1 reason = Timeout
4412025-06-03T00:51:10.507ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 1 reason = Timeout
4422025-06-03T00:51:10.507ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
4432025-06-03T00:51:10.511ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4442025-06-03T00:51:10.511ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
4452025-06-03T00:51:10.511ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4462025-06-03T00:51:10.511ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
4472025-06-03T00:51:10.511ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
4482025-06-03T00:51:10.511ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
4492025-06-03T00:51:10.511ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
4502025-06-03T00:51:10.515ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4512025-06-03T00:51:10.516ZDEBGnexus (service-firewall-rule-progator): starting background task for service firewall rule propagation background_task = service_firewall_rule_propagation
4522025-06-03T00:51:10.518ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
4532025-06-03T00:51:10.518ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
4542025-06-03T00:51:10.519ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 125.803\u{b5}s iteration = 1 status = {"error":"no config nor servers"}
4552025-06-03T00:51:10.519ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 27.531\u{b5}s iteration = 1 status = {"error":"no config nor servers"}
4562025-06-03T00:51:10.519ZDEBGnexus (BackgroundTasks): activating background_task = saga_recovery iteration = 1 reason = Timeout
4572025-06-03T00:51:10.519ZDEBGnexus (SagaRecoverer): listing candidate sagas for recovery
4582025-06-03T00:51:10.519ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 1 reason = Timeout
4592025-06-03T00:51:10.519ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 1 reason = Timeout
4602025-06-03T00:51:10.519ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 1 reason = Timeout
4612025-06-03T00:51:10.519ZDEBGnexus (BackgroundTasks): activating background_task = tuf_artifact_replication iteration = 1 reason = Timeout
4622025-06-03T00:51:10.519ZDEBGnexus (SagaRecoverer): roles roles = RoleSet { roles: {} }
4632025-06-03T00:51:10.519ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
4642025-06-03T00:51:10.519ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
4652025-06-03T00:51:10.519ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 1 reason = Timeout
4662025-06-03T00:51:10.519ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
4672025-06-03T00:51:10.519ZDEBGnexus (BackgroundTasks): activating background_task = webhook_deliverator iteration = 1 reason = Timeout
4682025-06-03T00:51:10.519ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 1 reason = Timeout
4692025-06-03T00:51:10.519ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
4702025-06-03T00:51:10.519ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 1 reason = Timeout
4712025-06-03T00:51:10.519ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 1 reason = Timeout
4722025-06-03T00:51:10.519ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
4732025-06-03T00:51:10.519ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_rendezvous iteration = 1 reason = Timeout
4742025-06-03T00:51:10.519ZDEBGnexus (BackgroundTasks): activating background_task = crdb_node_id_collector iteration = 1 reason = Timeout
4752025-06-03T00:51:10.519ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
4762025-06-03T00:51:10.519ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = crdb_node_id_collector file = nexus/src/app/background/tasks/crdb_node_id_collector.rs:78 reason = no blueprint
4772025-06-03T00:51:10.519ZWARNnexus (BackgroundTasks): Blueprint rendezvous: skipped background_task = blueprint_rendezvous file = nexus/src/app/background/tasks/blueprint_rendezvous.rs:52 reason = no blueprint
4782025-06-03T00:51:10.519ZDEBGnexus (BackgroundTasks): activation complete background_task = crdb_node_id_collector elapsed = 12.2372ms iteration = 1 status = {"error":"no blueprint"}
4792025-06-03T00:51:10.519ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_rendezvous elapsed = 12.247771ms iteration = 1 status = {"error":"no blueprint"}
4802025-06-03T00:51:10.519ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
4812025-06-03T00:51:10.519ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
4822025-06-03T00:51:10.520ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
4832025-06-03T00:51:10.520ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
4842025-06-03T00:51:10.520ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
4852025-06-03T00:51:10.521ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
4862025-06-03T00:51:10.522ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4872025-06-03T00:51:10.522ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
4882025-06-03T00:51:10.522ZDEBGnexus (SagaRecoverer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000003a8a, .. }) resource = Database result = Ok(())
4892025-06-03T00:51:10.522ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
4902025-06-03T00:51:10.522ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
4912025-06-03T00:51:10.522ZDEBGnexus (BackgroundTasks): roles background_task = webhook_deliverator roles = RoleSet { roles: {} }
4922025-06-03T00:51:10.523ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 1 reason = Timeout
4932025-06-03T00:51:10.523ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:76 reason = no blueprint
4942025-06-03T00:51:10.523ZDEBGnexus (BackgroundTasks): activating background_task = alert_dispatcher iteration = 1 reason = Timeout
4952025-06-03T00:51:10.523ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
4962025-06-03T00:51:10.523ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 15.814361ms iteration = 1 status = {"error":"no blueprint"}
4972025-06-03T00:51:10.523ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4982025-06-03T00:51:10.523ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
4992025-06-03T00:51:10.523ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
5002025-06-03T00:51:10.523ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
5012025-06-03T00:51:10.523ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
5022025-06-03T00:51:10.523ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
5032025-06-03T00:51:10.525ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("9feb16c9-ff60-43e8-917b-965124f46e0d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("aa124775-31de-4648-981a-4af500268b66.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("aa124775-31de-4648-981a-4af500268b66.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::9))) }], valid_until: Instant { tv_sec: 2397, tv_nsec: 748533843 } })
5042025-06-03T00:51:10.525ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("241573bc-467c-4802-9f4b-f7c82da4e12e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("64fc0a69-d4b8-4f6c-9a07-77a4cfa94962.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("ceb74144-4b6b-4b3f-bc80-eac2d75f3bc9.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("ceb74144-4b6b-4b3f-bc80-eac2d75f3bc9.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:2::1))) }], valid_until: Instant { tv_sec: 2397, tv_nsec: 748531603 } })
5052025-06-03T00:51:10.525ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
5062025-06-03T00:51:10.526ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
5072025-06-03T00:51:10.527ZINFOnexus (BackgroundTasks): found DNS servers (initial) addresses = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } background_task = dns_servers_internal dns_group = internal file = nexus/src/app/background/tasks/dns_servers.rs:113
5082025-06-03T00:51:10.527ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 20.548358ms iteration = 1 status = {"addresses":["[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:3::1]:5353","[fd00:1122:3344:2::1]:5353"]}
5092025-06-03T00:51:10.527ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 2 reason = Dependency
5102025-06-03T00:51:10.527ZWARNnexus (BackgroundTasks): DNS propagation: skipped background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:69 reason = no config
5112025-06-03T00:51:10.527ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 7.58\u{b5}s iteration = 2 status = {"error":"no config"}
5122025-06-03T00:51:10.527ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
5132025-06-03T00:51:10.527ZINFOnexus (BackgroundTasks): found DNS servers (initial) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external dns_group = external file = nexus/src/app/background/tasks/dns_servers.rs:113
5142025-06-03T00:51:10.527ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 20.718283ms iteration = 1 status = {"addresses":["[fd00:1122:3344:101::8]:5353","[fd00:1122:3344:101::9]:5353"]}
5152025-06-03T00:51:10.527ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 2 reason = Dependency
5162025-06-03T00:51:10.527ZWARNnexus (BackgroundTasks): DNS propagation: skipped background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:69 reason = no config
5172025-06-03T00:51:10.527ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 6.4\u{b5}s iteration = 2 status = {"error":"no config"}
5182025-06-03T00:51:10.527ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
5192025-06-03T00:51:10.527ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
5202025-06-03T00:51:10.527ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
5212025-06-03T00:51:10.527ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
5222025-06-03T00:51:10.528ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
5232025-06-03T00:51:10.528ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
5242025-06-03T00:51:10.528ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
5252025-06-03T00:51:10.528ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
5262025-06-03T00:51:10.529ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
5272025-06-03T00:51:10.529ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
5282025-06-03T00:51:10.529ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
5292025-06-03T00:51:10.529ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
5302025-06-03T00:51:10.529ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
5312025-06-03T00:51:10.529ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
5322025-06-03T00:51:10.529ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
5332025-06-03T00:51:10.529ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = alert_dispatcher resource = Database result = Ok(())
5342025-06-03T00:51:10.529ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
5352025-06-03T00:51:10.529ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5362025-06-03T00:51:10.529ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
5372025-06-03T00:51:10.530ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
5382025-06-03T00:51:10.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
5392025-06-03T00:51:10.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = webhook_deliverator resource = Database result = Ok(())
5402025-06-03T00:51:10.533ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
5412025-06-03T00:51:10.533ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
5422025-06-03T00:51:10.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
5432025-06-03T00:51:10.538ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 32.174579ms iteration = 1 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
5442025-06-03T00:51:10.549ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5452025-06-03T00:51:10.553ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
5462025-06-03T00:51:10.553ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
5472025-06-03T00:51:10.554ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
5482025-06-03T00:51:10.573ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5492025-06-03T00:51:10.573ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-06-03 00:41:10.506877200 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
5502025-06-03T00:51:10.574ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 67.771455ms iteration = 1 status = {"expiration":"2025-06-03T00:41:10.506877200Z","pruned":[]}
5512025-06-03T00:51:10.577ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Fleet result = Ok(())
5522025-06-03T00:51:10.577ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
5532025-06-03T00:51:10.577ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
5542025-06-03T00:51:10.581ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 73.881574ms iteration = 1 status = {}
5552025-06-03T00:51:10.583ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 76.153705ms iteration = 1 status = {"errors":[],"found_port_ok":[]}
5562025-06-03T00:51:10.601ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-afe89ed7-19b7-479b-81eb-05c988ebc66d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-afe89ed7-19b7-479b-81eb-05c988ebc66d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 2397, tv_nsec: 824939305 } })
5572025-06-03T00:51:10.602ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1179
5582025-06-03T00:51:10.608ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-afe89ed7-19b7-479b-81eb-05c988ebc66d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-afe89ed7-19b7-479b-81eb-05c988ebc66d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 2397, tv_nsec: 831412577 } })
5592025-06-03T00:51:10.608ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1179
5602025-06-03T00:51:10.652ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
5612025-06-03T00:51:10.653ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
5622025-06-03T00:51:10.669ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:1025 local_addr = [fd00:1122:3344:101::a]:12221 remote_addr = [fd00:1122:3344:101::d]:48912
5632025-06-03T00:51:10.680ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 173.034543ms iteration = 1 status = {"errors":[],"finish_invoked_ok":[]}
5642025-06-03T00:51:10.681ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5652025-06-03T00:51:10.686ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5662025-06-03T00:51:10.686ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:48912 req_id = b0983332-1ad7-4838-a683-5fe63031686c roles = RoleSet { roles: {} } uri = /metrics/collectors/292a19e1-4305-4e49-b11e-08ac74714bf3/producers?sort_by=id_ascending
5672025-06-03T00:51:10.687ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
5682025-06-03T00:51:10.687ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
5692025-06-03T00:51:10.687ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:48912 req_id = b0983332-1ad7-4838-a683-5fe63031686c resource = Database result = Ok(()) uri = /metrics/collectors/292a19e1-4305-4e49-b11e-08ac74714bf3/producers?sort_by=id_ascending
5702025-06-03T00:51:10.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
5712025-06-03T00:51:10.689ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
5722025-06-03T00:51:10.689ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5732025-06-03T00:51:10.690ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
5742025-06-03T00:51:10.691ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
5752025-06-03T00:51:10.691ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
5762025-06-03T00:51:10.691ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
5772025-06-03T00:51:10.691ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
5782025-06-03T00:51:10.692ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5792025-06-03T00:51:10.692ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
5802025-06-03T00:51:10.693ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5812025-06-03T00:51:10.693ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
5822025-06-03T00:51:10.693ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
5832025-06-03T00:51:10.693ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
5842025-06-03T00:51:10.695ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
5852025-06-03T00:51:10.695ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
5862025-06-03T00:51:10.695ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
5872025-06-03T00:51:10.698ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
5882025-06-03T00:51:10.698ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
5892025-06-03T00:51:10.698ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
5902025-06-03T00:51:10.698ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
5912025-06-03T00:51:10.698ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
5922025-06-03T00:51:10.702ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1187 zone_address = fd00:1122:3344:101::2
5932025-06-03T00:51:10.705ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
5942025-06-03T00:51:10.705ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "66a767b1-6923-4cc1-aad9-01dc665e82d8", "content-length": "26", "date": "Tue, 03 Jun 2025 00:51:10 GMT"} })
5952025-06-03T00:51:10.706ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1190 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
5962025-06-03T00:51:10.706ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
5972025-06-03T00:51:10.706ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
5982025-06-03T00:51:10.707ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 199.690791ms iteration = 1 status = {"errors":[],"garbage_collect_requested":[]}
5992025-06-03T00:51:10.707ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1224 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
6002025-06-03T00:51:10.710ZINFOnexus (BackgroundTasks): all webhook delivery tasks completed background_task = webhook_deliverator file = nexus/src/app/background/tasks/webhook_deliverator.rs:202 num_receivers = 0
6012025-06-03T00:51:10.710ZDEBGnexus (BackgroundTasks): activation complete background_task = webhook_deliverator elapsed = 203.124607ms iteration = 1 status = {"by_rx":{},"error":null}
6022025-06-03T00:51:10.719ZINFOnexus (BackgroundTasks): Physical Disk Adoption: Rack not yet initialized background_task = physical_disk_adoption file = nexus/src/app/background/tasks/physical_disk_adoption.rs:81 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae
6032025-06-03T00:51:10.719ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 211.771486ms iteration = 1 status = {"error":"rack not yet initialized: d6831ba0-55f0-4191-8dc2-ea15e55ab0ae"}
6042025-06-03T00:51:10.719ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1187 zone_address = fd00:1122:3344:101::2
6052025-06-03T00:51:10.719ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
6062025-06-03T00:51:10.720ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "6967c88e-bc24-4e48-adcf-40ee45538609", "content-length": "26", "date": "Tue, 03 Jun 2025 00:51:10 GMT"} })
6072025-06-03T00:51:10.720ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1190 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
6082025-06-03T00:51:10.722ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1224 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
6092025-06-03T00:51:10.725ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6102025-06-03T00:51:10.728ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
6112025-06-03T00:51:10.728ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
6122025-06-03T00:51:10.728ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
6132025-06-03T00:51:10.729ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6142025-06-03T00:51:10.731ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
6152025-06-03T00:51:10.731ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = alert_dispatcher resource = Database result = Ok(())
6162025-06-03T00:51:10.732ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
6172025-06-03T00:51:10.733ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
6182025-06-03T00:51:10.733ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
6192025-06-03T00:51:10.734ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 57598 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:48912 req_id = b0983332-1ad7-4838-a683-5fe63031686c response_code = 200 uri = /metrics/collectors/292a19e1-4305-4e49-b11e-08ac74714bf3/producers?sort_by=id_ascending
6202025-06-03T00:51:10.737ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
6212025-06-03T00:51:10.738ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
6222025-06-03T00:51:10.739ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
6232025-06-03T00:51:10.739ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
6242025-06-03T00:51:10.742ZINFOnexus (SagaRecoverer): listed in-progress sagas count = 0 file = nexus/src/app/background/tasks/saga_recovery.rs:438
6252025-06-03T00:51:10.742ZDEBGnexus (BackgroundTasks): activation complete background_task = saga_recovery elapsed = 235.125541ms iteration = 1 status = {"last_pass":{"Success":{"nfailed":0,"nfound":0,"nrecovered":0,"nremoved":0,"nskipped":0}},"ntotal_failures":0,"ntotal_finished":0,"ntotal_recovered":0,"ntotal_sec_errors_bad_state":0,"ntotal_sec_errors_missing":0,"ntotal_started":0,"recent_failures":[],"recent_recoveries":[]}
6262025-06-03T00:51:10.742ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6272025-06-03T00:51:10.747ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
6282025-06-03T00:51:10.749ZWARNnexus (BackgroundTasks): inventory collection is None background_task = service_zone_nat_tracker file = nexus/src/app/background/tasks/sync_service_zone_nat.rs:92
6292025-06-03T00:51:10.749ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 241.84383ms iteration = 1 status = {"error":"inventory collection is None"}
6302025-06-03T00:51:10.754ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
6312025-06-03T00:51:10.755ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
6322025-06-03T00:51:10.756ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6332025-06-03T00:51:10.761ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
6342025-06-03T00:51:10.761ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6352025-06-03T00:51:10.761ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6362025-06-03T00:51:10.762ZDEBGnexus (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = []
6372025-06-03T00:51:10.762ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
6382025-06-03T00:51:10.763ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-afe89ed7-19b7-479b-81eb-05c988ebc66d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-afe89ed7-19b7-479b-81eb-05c988ebc66d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 2397, tv_nsec: 986312307 } })
6392025-06-03T00:51:10.763ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
6402025-06-03T00:51:10.765ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
6412025-06-03T00:51:10.765ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
6422025-06-03T00:51:10.766ZWARNnexus (BackgroundTasks): failed to read target blueprint background_task = blueprint_loader error = Internal Error: no target blueprint set file = nexus/src/app/background/tasks/blueprint_load.rs:69
6432025-06-03T00:51:10.766ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 260.082006ms iteration = 1 status = {"error":"failed to read target blueprint: Internal Error: no target blueprint set"}
6442025-06-03T00:51:10.767ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6452025-06-03T00:51:10.768ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
6462025-06-03T00:51:10.768ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
6472025-06-03T00:51:10.772ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
6482025-06-03T00:51:10.772ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
6492025-06-03T00:51:10.772ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
6502025-06-03T00:51:10.773ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
6512025-06-03T00:51:10.774ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
6522025-06-03T00:51:10.776ZDEBGnexus (BackgroundTasks): activation complete background_task = alert_dispatcher elapsed = 269.170998ms iteration = 1 status = {"dispatched":[],"errors":[],"glob_version":"144.0.0","globs_reprocessed":{},"no_receivers":[]}
6532025-06-03T00:51:10.777ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6542025-06-03T00:51:10.779ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 272.049287ms iteration = 1 status = {"errors":[],"requests_created_ok":[]}
6552025-06-03T00:51:10.783ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
6562025-06-03T00:51:10.783ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
6572025-06-03T00:51:10.783ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
6582025-06-03T00:51:10.786ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
6592025-06-03T00:51:10.787ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
6602025-06-03T00:51:10.789ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
6612025-06-03T00:51:10.789ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
6622025-06-03T00:51:10.789ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:167 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
6632025-06-03T00:51:10.789ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 282.218373ms iteration = 1 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
6642025-06-03T00:51:10.789ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
6652025-06-03T00:51:10.791ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6662025-06-03T00:51:10.802ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 295.61949ms iteration = 1 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
6672025-06-03T00:51:10.803ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
6682025-06-03T00:51:10.803ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
6692025-06-03T00:51:10.803ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
6702025-06-03T00:51:10.805ZWARNnexus (BackgroundTasks): failed to read DNS config background_task = dns_config_internal dns_group = internal error = Internal Error: expected exactly one latest version for DNS group Internal, found 0 file = nexus/src/app/background/tasks/dns_config.rs:72
6712025-06-03T00:51:10.805ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 299.734517ms iteration = 1 status = {"error":"failed to read DNS config: Internal Error: expected exactly one latest version for DNS group Internal, found 0"}
6722025-06-03T00:51:10.806ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6732025-06-03T00:51:10.806ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
6742025-06-03T00:51:10.807ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
6752025-06-03T00:51:10.807ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
6762025-06-03T00:51:10.807ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
6772025-06-03T00:51:10.807ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:137 found = 0 not_ready_to_be_deleted = 0
6782025-06-03T00:51:10.807ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 300.707678ms iteration = 1 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
6792025-06-03T00:51:10.807ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
6802025-06-03T00:51:10.809ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Fleet result = Ok(())
6812025-06-03T00:51:10.809ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
6822025-06-03T00:51:10.809ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
6832025-06-03T00:51:10.809ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
6842025-06-03T00:51:10.810ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
6852025-06-03T00:51:10.811ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
6862025-06-03T00:51:10.811ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
6872025-06-03T00:51:10.811ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
6882025-06-03T00:51:10.811ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
6892025-06-03T00:51:10.814ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
6902025-06-03T00:51:10.814ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
6912025-06-03T00:51:10.817ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
6922025-06-03T00:51:10.817ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
6932025-06-03T00:51:10.820ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
6942025-06-03T00:51:10.821ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
6952025-06-03T00:51:10.828ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:547 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
6962025-06-03T00:51:10.828ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 321.42901ms iteration = 1 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
6972025-06-03T00:51:10.830ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6982025-06-03T00:51:10.830ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
6992025-06-03T00:51:10.830ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
7002025-06-03T00:51:10.833ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
7012025-06-03T00:51:10.833ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
7022025-06-03T00:51:10.833ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
7032025-06-03T00:51:10.838ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7042025-06-03T00:51:10.839ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
7052025-06-03T00:51:10.840ZWARNnexus (BackgroundTasks): failed to read DNS config background_task = dns_config_external dns_group = external error = Internal Error: expected exactly one latest version for DNS group External, found 0 file = nexus/src/app/background/tasks/dns_config.rs:72
7062025-06-03T00:51:10.840ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 333.173336ms iteration = 1 status = {"error":"failed to read DNS config: Internal Error: expected exactly one latest version for DNS group External, found 0"}
7072025-06-03T00:51:10.840ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
7082025-06-03T00:51:10.840ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
7092025-06-03T00:51:10.840ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
7102025-06-03T00:51:10.840ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
7112025-06-03T00:51:10.842ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7122025-06-03T00:51:10.844ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 337.546881ms iteration = 1 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
7132025-06-03T00:51:10.848ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
7142025-06-03T00:51:10.849ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
7152025-06-03T00:51:10.850ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
7162025-06-03T00:51:10.858ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 351.785073ms iteration = 1 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
7172025-06-03T00:51:10.858ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
7182025-06-03T00:51:10.858ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
7192025-06-03T00:51:10.859ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
7202025-06-03T00:51:10.859ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
7212025-06-03T00:51:10.864ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
7222025-06-03T00:51:10.864ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
7232025-06-03T00:51:10.872ZWARNnexus (BackgroundTasks): failed to read Silo/DNS/TLS configuration background_task = external_endpoints error = Internal Error: expected at least one external DNS zone file = nexus/src/app/background/tasks/external_endpoints.rs:48
7242025-06-03T00:51:10.872ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 365.540201ms iteration = 1 status = {"error":"failed to read Silo/DNS/TLS configuration: Internal Error: expected at least one external DNS zone"}
7252025-06-03T00:51:10.875ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 367.274454ms iteration = 1 status = {}
7262025-06-03T00:51:10.878ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
7272025-06-03T00:51:10.878ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
7282025-06-03T00:51:10.883ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
7292025-06-03T00:51:10.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
7302025-06-03T00:51:10.886ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
7312025-06-03T00:51:10.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
7322025-06-03T00:51:10.889ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
7332025-06-03T00:51:10.890ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
7342025-06-03T00:51:10.890ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
7352025-06-03T00:51:10.890ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
7362025-06-03T00:51:10.896ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 389.33591ms iteration = 1 status = null
7372025-06-03T00:51:10.897ZDEBGnexus (BackgroundTasks): begin collection background_task = inventory_collection
7382025-06-03T00:51:10.898ZDEBGnexus (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[fd00:1122:3344:101::2]:12225
7392025-06-03T00:51:10.898ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/ignition
7402025-06-03T00:51:10.898ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "62d51b86-ca7e-47bb-8fa4-b925b62e9dc5", "content-length": "476", "date": "Tue, 03 Jun 2025 00:51:10 GMT"} })
7412025-06-03T00:51:10.901ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0
7422025-06-03T00:51:10.902ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "9661ae77-7fb3-4b27-91c9-ca257117e2fe", "content-length": "734", "date": "Tue, 03 Jun 2025 00:51:10 GMT"} })
7432025-06-03T00:51:10.903ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0
7442025-06-03T00:51:10.903ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "92c3f16d-2658-4be9-8150-6488e05950f2", "content-length": "111", "date": "Tue, 03 Jun 2025 00:51:10 GMT"} })
7452025-06-03T00:51:10.903ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1
7462025-06-03T00:51:10.904ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "1153bc3b-8c65-4aee-8683-d34a5c856d0d", "content-length": "111", "date": "Tue, 03 Jun 2025 00:51:10 GMT"} })
7472025-06-03T00:51:10.904ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0
7482025-06-03T00:51:10.904ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ac065a15-b7fd-491e-bc40-e33137497bdd", "content-length": "170", "date": "Tue, 03 Jun 2025 00:51:10 GMT"} })
7492025-06-03T00:51:10.904ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1
7502025-06-03T00:51:10.905ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c550c7ff-f640-49f0-a6a0-52eeccdd3874", "content-length": "170", "date": "Tue, 03 Jun 2025 00:51:10 GMT"} })
7512025-06-03T00:51:10.905ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0
7522025-06-03T00:51:10.905ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ce71edaf-e51b-4361-9382-aade32ac0285", "content-length": "179", "date": "Tue, 03 Jun 2025 00:51:10 GMT"} })
7532025-06-03T00:51:10.905ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1
7542025-06-03T00:51:10.905ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 398.67261ms iteration = 1 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
7552025-06-03T00:51:10.905ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "461f8acc-5ceb-48bd-ba2f-303f123ceed7", "content-length": "179", "date": "Tue, 03 Jun 2025 00:51:10 GMT"} })
7562025-06-03T00:51:10.906ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa
7572025-06-03T00:51:10.906ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "40e45ebc-4982-4d26-b4a1-fba61d7e7f03", "content-length": "702", "date": "Tue, 03 Jun 2025 00:51:10 GMT"} })
7582025-06-03T00:51:10.906ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
7592025-06-03T00:51:10.907ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "4c93149a-efd8-445e-8e7a-6b59f83de36d", "content-length": "727", "date": "Tue, 03 Jun 2025 00:51:10 GMT"} })
7602025-06-03T00:51:10.907ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
7612025-06-03T00:51:10.907ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "1cc943fa-8194-44d9-93d7-f6420b93cc63", "content-length": "729", "date": "Tue, 03 Jun 2025 00:51:10 GMT"} })
7622025-06-03T00:51:10.907ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
7632025-06-03T00:51:10.907ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f8f9538b-84c8-479b-983e-c6905bf92c2c", "content-length": "728", "date": "Tue, 03 Jun 2025 00:51:10 GMT"} })
7642025-06-03T00:51:10.907ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0
7652025-06-03T00:51:10.908ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "f814dd7f-8eec-4ce6-924e-bf0948f1fa27", "content-length": "722", "date": "Tue, 03 Jun 2025 00:51:10 GMT"} })
7662025-06-03T00:51:10.908ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0
7672025-06-03T00:51:10.908ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "cbb44157-3c57-47c8-a18c-e049b2f648fa", "content-length": "109", "date": "Tue, 03 Jun 2025 00:51:10 GMT"} })
7682025-06-03T00:51:10.908ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1
7692025-06-03T00:51:10.908ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "79ce5190-5155-4773-9651-0c9015306786", "content-length": "109", "date": "Tue, 03 Jun 2025 00:51:10 GMT"} })
7702025-06-03T00:51:10.908ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0
7712025-06-03T00:51:10.909ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "393a1785-d8a9-482a-ba44-bfc44e53ce7f", "content-length": "169", "date": "Tue, 03 Jun 2025 00:51:10 GMT"} })
7722025-06-03T00:51:10.909ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1
7732025-06-03T00:51:10.909ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "0a24661f-bfab-46cf-892d-7338b38b6613", "content-length": "169", "date": "Tue, 03 Jun 2025 00:51:10 GMT"} })
7742025-06-03T00:51:10.909ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0
7752025-06-03T00:51:10.910ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "bd15a833-e13e-4f36-b14c-464d138a1847", "content-length": "178", "date": "Tue, 03 Jun 2025 00:51:10 GMT"} })
7762025-06-03T00:51:10.910ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1
7772025-06-03T00:51:10.910ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "d2ffb7fb-8308-4746-9d9d-1a2a4302e0f5", "content-length": "178", "date": "Tue, 03 Jun 2025 00:51:10 GMT"} })
7782025-06-03T00:51:10.910ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa
7792025-06-03T00:51:10.910ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "7d83b984-384c-4308-9e4b-c308c251fbc6", "content-length": "702", "date": "Tue, 03 Jun 2025 00:51:10 GMT"} })
7802025-06-03T00:51:10.910ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
7812025-06-03T00:51:10.910ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c96fb556-d6dc-48e4-9405-05f19a917c32", "content-length": "727", "date": "Tue, 03 Jun 2025 00:51:10 GMT"} })
7822025-06-03T00:51:10.910ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
7832025-06-03T00:51:10.911ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "10de1978-87ba-41ab-940f-aaf9038ceb15", "content-length": "729", "date": "Tue, 03 Jun 2025 00:51:10 GMT"} })
7842025-06-03T00:51:10.911ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
7852025-06-03T00:51:10.911ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "678cbf50-0a55-4f31-ac5a-8b315f891caa", "content-length": "728", "date": "Tue, 03 Jun 2025 00:51:10 GMT"} })
7862025-06-03T00:51:10.911ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
7872025-06-03T00:51:10.911ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
7882025-06-03T00:51:10.914ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7892025-06-03T00:51:10.915ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
7902025-06-03T00:51:10.915ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
7912025-06-03T00:51:10.915ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 408.381311ms iteration = 1 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
7922025-06-03T00:51:10.917ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
7932025-06-03T00:51:10.917ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
7942025-06-03T00:51:10.917ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
7952025-06-03T00:51:10.920ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:372
7962025-06-03T00:51:10.920ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 413.82665ms iteration = 1 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
7972025-06-03T00:51:10.943ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
7982025-06-03T00:51:10.944ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
7992025-06-03T00:51:10.944ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
8002025-06-03T00:51:10.944ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
8012025-06-03T00:51:10.945ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
8022025-06-03T00:51:10.946ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8032025-06-03T00:51:10.946ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
8042025-06-03T00:51:10.948ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8052025-06-03T00:51:10.951ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
8062025-06-03T00:51:10.951ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
8072025-06-03T00:51:10.952ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
8082025-06-03T00:51:10.952ZDEBGnexus (ServiceBalancer): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Rack { parent: Fleet, key: d6831ba0-55f0-4191-8dc2-ea15e55ab0ae, lookup_type: ById(d6831ba0-55f0-4191-8dc2-ea15e55ab0ae) } result = Ok(())
8092025-06-03T00:51:10.952ZINFOnexus: Still waiting for rack initialization: Rack { identity: RackIdentity { id: d6831ba0-55f0-4191-8dc2-ea15e55ab0ae, time_created: 2025-06-03T00:51:10.288786Z, time_modified: 2025-06-03T00:51:10.288786Z }, initialized: false, tuf_base_url: None, rack_subnet: Some(V6(Ipv6Network { addr: fd00:1122:3344:100::, prefix: 56 })) } file = nexus/src/app/rack.rs:775
8102025-06-03T00:51:10.967ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
8112025-06-03T00:51:10.967ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 459.91335ms iteration = 1 status = {}
8122025-06-03T00:51:10.968ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
8132025-06-03T00:51:10.970ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8142025-06-03T00:51:10.972ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Fleet result = Ok(())
8152025-06-03T00:51:10.972ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
8162025-06-03T00:51:10.972ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
8172025-06-03T00:51:10.973ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
8182025-06-03T00:51:10.974ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
8192025-06-03T00:51:10.974ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8202025-06-03T00:51:10.977ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
8212025-06-03T00:51:10.977ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
8222025-06-03T00:51:10.977ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
8232025-06-03T00:51:11.009ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
8242025-06-03T00:51:11.009ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
8252025-06-03T00:51:11.014ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
8262025-06-03T00:51:11.015ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
8272025-06-03T00:51:11.017ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8282025-06-03T00:51:11.021ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
8292025-06-03T00:51:11.021ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
8302025-06-03T00:51:11.021ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
8312025-06-03T00:51:11.025ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8322025-06-03T00:51:11.026ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
8332025-06-03T00:51:11.026ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
8342025-06-03T00:51:11.028ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Fleet result = Ok(())
8352025-06-03T00:51:11.028ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
8362025-06-03T00:51:11.028ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
8372025-06-03T00:51:11.028ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
8382025-06-03T00:51:11.028ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
8392025-06-03T00:51:11.033ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
8402025-06-03T00:51:11.034ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
8412025-06-03T00:51:11.034ZDEBGnexus (BackgroundTasks): activation complete background_task = tuf_artifact_replication elapsed = 527.50237ms iteration = 1 status = {"generation":1,"last_run_counters":{"copy_err":0,"copy_ok":0,"list_err":0,"list_ok":0,"put_config_err":0,"put_config_ok":0,"put_err":0,"put_ok":0},"lifetime_counters":{"copy_err":0,"copy_ok":0,"list_err":0,"list_ok":0,"put_config_err":0,"put_config_ok":0,"put_err":0,"put_ok":0},"local_repos":0,"request_debug_ringbuf":[]}
8422025-06-03T00:51:11.039ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
8432025-06-03T00:51:11.039ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
8442025-06-03T00:51:11.042ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
8452025-06-03T00:51:11.043ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
8462025-06-03T00:51:11.070ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
8472025-06-03T00:51:11.070ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
8482025-06-03T00:51:11.072ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8492025-06-03T00:51:11.074ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
8502025-06-03T00:51:11.074ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
8512025-06-03T00:51:11.074ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
8522025-06-03T00:51:11.075ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8532025-06-03T00:51:11.077ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
8542025-06-03T00:51:11.077ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
8552025-06-03T00:51:11.077ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
8562025-06-03T00:51:11.078ZINFOnexus (BackgroundTasks): instance reincarnation completed successfully background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:113 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0
8572025-06-03T00:51:11.078ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 571.372112ms iteration = 1 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
8582025-06-03T00:51:11.080ZDEBGnexus (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
8592025-06-03T00:51:11.080ZDEBGnexus (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
8602025-06-03T00:51:11.080ZDEBGnexus (BackgroundTasks): finished collection background_task = inventory_collection
8612025-06-03T00:51:11.081ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
8622025-06-03T00:51:11.081ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
8632025-06-03T00:51:11.083ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8642025-06-03T00:51:11.085ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
8652025-06-03T00:51:11.085ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
8662025-06-03T00:51:11.085ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
8672025-06-03T00:51:11.091ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
8682025-06-03T00:51:11.091ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
8692025-06-03T00:51:11.092ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
8702025-06-03T00:51:11.093ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
8712025-06-03T00:51:11.102ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8722025-06-03T00:51:11.107ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
8732025-06-03T00:51:11.107ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
8742025-06-03T00:51:11.107ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
8752025-06-03T00:51:11.109ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
8762025-06-03T00:51:11.110ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
8772025-06-03T00:51:11.112ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
8782025-06-03T00:51:11.112ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
8792025-06-03T00:51:11.115ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8802025-06-03T00:51:11.120ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
8812025-06-03T00:51:11.120ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
8822025-06-03T00:51:11.121ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
8832025-06-03T00:51:11.129ZDEBGnexus (service-firewall-rule-progator): resolved names for firewall rules background_task = service_firewall_rule_propagation instance_interfaces = {} subnet_interfaces = {} subnet_networks = {Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })], Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })]} vpc_interfaces = {}
8842025-06-03T00:51:11.130ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
8852025-06-03T00:51:11.130ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
8862025-06-03T00:51:11.135ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8872025-06-03T00:51:11.137ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Fleet result = Ok(())
8882025-06-03T00:51:11.137ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
8892025-06-03T00:51:11.137ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
8902025-06-03T00:51:11.139ZDEBGnexus (service-firewall-rule-progator): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified. background_task = service_firewall_rule_propagation
8912025-06-03T00:51:11.139ZDEBGnexus (service-firewall-rule-progator): resolved firewall rules for sled agents background_task = service_firewall_rule_propagation sled_agent_rules = []
8922025-06-03T00:51:11.139ZDEBGnexus (service-firewall-rule-progator): resolved 0 rules for sleds background_task = service_firewall_rule_propagation
8932025-06-03T00:51:11.166ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 659.779616ms iteration = 1 status = {}
8942025-06-03T00:51:11.238ZDEBGnexus (service-firewall-rule-progator): resolved sleds for vpc oxide-services background_task = service_firewall_rule_propagation vpc_to_sled = []
8952025-06-03T00:51:11.238ZDEBGnexus (service-firewall-rule-progator): sending firewall rules to sled agents background_task = service_firewall_rule_propagation
8962025-06-03T00:51:11.238ZINFOnexus (service-firewall-rule-progator): updated firewall rules on 0 sleds background_task = service_firewall_rule_propagation file = nexus/networking/src/firewall_rules.rs:495
8972025-06-03T00:51:11.239ZDEBGnexus (service-firewall-rule-progator): successfully propagated service firewall rules background_task = service_firewall_rule_propagation elapsed = 722.831054ms
8982025-06-03T00:51:11.239ZDEBGnexus (BackgroundTasks): activation complete background_task = service_firewall_rule_propagation elapsed = 731.969228ms iteration = 1 status = {}
8992025-06-03T00:51:11.412ZINFOnexus (BackgroundTasks): inserted inventory collection background_task = inventory_collection collection_id = 347e452d-2d28-4a95-a97e-9e9997a4b5de file = nexus/db-queries/src/db/datastore/inventory.rs:1192
9002025-06-03T00:51:11.412ZDEBGnexus (BackgroundTasks): inventory collection complete background_task = inventory_collection collection_id = 347e452d-2d28-4a95-a97e-9e9997a4b5de time_started = 2025-06-03 00:51:10.894608 UTC
9012025-06-03T00:51:11.412ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_rendezvous iteration = 2 reason = Dependency
9022025-06-03T00:51:11.412ZWARNnexus (BackgroundTasks): Blueprint rendezvous: skipped background_task = blueprint_rendezvous file = nexus/src/app/background/tasks/blueprint_rendezvous.rs:52 reason = no blueprint
9032025-06-03T00:51:11.412ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_rendezvous elapsed = 27.421\u{b5}s iteration = 2 status = {"error":"no blueprint"}
9042025-06-03T00:51:11.412ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 904.98564ms iteration = 1 status = {"collection_id":"347e452d-2d28-4a95-a97e-9e9997a4b5de","time_done":"2025-06-03 00:51:11.080308 UTC","time_started":"2025-06-03 00:51:10.894608 UTC"}
9052025-06-03T00:51:11.412ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 2 reason = Dependency
9062025-06-03T00:51:11.412ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
9072025-06-03T00:51:11.412ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
9082025-06-03T00:51:11.413ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9092025-06-03T00:51:11.415ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
9102025-06-03T00:51:11.415ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
9112025-06-03T00:51:11.415ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
9122025-06-03T00:51:11.417ZINFOnexus (BackgroundTasks): Physical Disk Adoption: Rack not yet initialized background_task = physical_disk_adoption file = nexus/src/app/background/tasks/physical_disk_adoption.rs:81 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae
9132025-06-03T00:51:11.417ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 5.587594ms iteration = 2 status = {"error":"rack not yet initialized: d6831ba0-55f0-4191-8dc2-ea15e55ab0ae"}
9142025-06-03T00:51:12.953ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
9152025-06-03T00:51:12.953ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
9162025-06-03T00:51:12.955ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
9172025-06-03T00:51:12.956ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
9182025-06-03T00:51:12.957ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9192025-06-03T00:51:12.960ZDEBGnexus (ServiceBalancer): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Rack { parent: Fleet, key: d6831ba0-55f0-4191-8dc2-ea15e55ab0ae, lookup_type: ById(d6831ba0-55f0-4191-8dc2-ea15e55ab0ae) } result = Ok(())
9202025-06-03T00:51:12.960ZINFOnexus: Still waiting for rack initialization: Rack { identity: RackIdentity { id: d6831ba0-55f0-4191-8dc2-ea15e55ab0ae, time_created: 2025-06-03T00:51:10.288786Z, time_modified: 2025-06-03T00:51:10.288786Z }, initialized: false, tuf_base_url: None, rack_subnet: Some(V6(Ipv6Network { addr: fd00:1122:3344:100::, prefix: 56 })) } file = nexus/src/app/rack.rs:775
9212025-06-03T00:51:14.960ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
9222025-06-03T00:51:14.961ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
9232025-06-03T00:51:14.962ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
9242025-06-03T00:51:14.963ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
9252025-06-03T00:51:14.964ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9262025-06-03T00:51:14.966ZDEBGnexus (ServiceBalancer): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Rack { parent: Fleet, key: d6831ba0-55f0-4191-8dc2-ea15e55ab0ae, lookup_type: ById(d6831ba0-55f0-4191-8dc2-ea15e55ab0ae) } result = Ok(())
9272025-06-03T00:51:14.966ZINFOnexus: Still waiting for rack initialization: Rack { identity: RackIdentity { id: d6831ba0-55f0-4191-8dc2-ea15e55ab0ae, time_created: 2025-06-03T00:51:10.288786Z, time_modified: 2025-06-03T00:51:10.288786Z }, initialized: false, tuf_base_url: None, rack_subnet: Some(V6(Ipv6Network { addr: fd00:1122:3344:100::, prefix: 56 })) } file = nexus/src/app/rack.rs:775
9282025-06-03T00:51:16.968ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
9292025-06-03T00:51:16.968ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
9302025-06-03T00:51:16.970ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
9312025-06-03T00:51:16.970ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
9322025-06-03T00:51:16.972ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9332025-06-03T00:51:16.974ZDEBGnexus (ServiceBalancer): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Rack { parent: Fleet, key: d6831ba0-55f0-4191-8dc2-ea15e55ab0ae, lookup_type: ById(d6831ba0-55f0-4191-8dc2-ea15e55ab0ae) } result = Ok(())
9342025-06-03T00:51:16.974ZINFOnexus: Still waiting for rack initialization: Rack { identity: RackIdentity { id: d6831ba0-55f0-4191-8dc2-ea15e55ab0ae, time_created: 2025-06-03T00:51:10.288786Z, time_modified: 2025-06-03T00:51:10.288786Z }, initialized: false, tuf_base_url: None, rack_subnet: Some(V6(Ipv6Network { addr: fd00:1122:3344:100::, prefix: 56 })) } file = nexus/src/app/rack.rs:775
9352025-06-03T00:51:18.976ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
9362025-06-03T00:51:18.976ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
9372025-06-03T00:51:18.978ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
9382025-06-03T00:51:18.978ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
9392025-06-03T00:51:18.980ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9402025-06-03T00:51:18.982ZDEBGnexus (ServiceBalancer): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Rack { parent: Fleet, key: d6831ba0-55f0-4191-8dc2-ea15e55ab0ae, lookup_type: ById(d6831ba0-55f0-4191-8dc2-ea15e55ab0ae) } result = Ok(())
9412025-06-03T00:51:18.982ZINFOnexus: Still waiting for rack initialization: Rack { identity: RackIdentity { id: d6831ba0-55f0-4191-8dc2-ea15e55ab0ae, time_created: 2025-06-03T00:51:10.288786Z, time_modified: 2025-06-03T00:51:10.288786Z }, initialized: false, tuf_base_url: None, rack_subnet: Some(V6(Ipv6Network { addr: fd00:1122:3344:100::, prefix: 56 })) } file = nexus/src/app/rack.rs:775
9422025-06-03T00:51:20.507ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 2 reason = Timeout
9432025-06-03T00:51:20.507ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
9442025-06-03T00:51:20.507ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 2 reason = Timeout
9452025-06-03T00:51:20.507ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9462025-06-03T00:51:20.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
9472025-06-03T00:51:20.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9482025-06-03T00:51:20.509ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9492025-06-03T00:51:20.512ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
9502025-06-03T00:51:20.512ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9512025-06-03T00:51:20.512ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9522025-06-03T00:51:20.515ZWARNnexus (BackgroundTasks): failed to read target blueprint background_task = blueprint_loader error = Internal Error: no target blueprint set file = nexus/src/app/background/tasks/blueprint_load.rs:69
9532025-06-03T00:51:20.515ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 8.012839ms iteration = 2 status = {"error":"failed to read target blueprint: Internal Error: no target blueprint set"}
9542025-06-03T00:51:20.539ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
9552025-06-03T00:51:20.539ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
9562025-06-03T00:51:20.575ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 68.529908ms iteration = 2 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
9572025-06-03T00:51:20.984ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
9582025-06-03T00:51:20.984ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
9592025-06-03T00:51:20.986ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
9602025-06-03T00:51:20.986ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
9612025-06-03T00:51:20.987ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9622025-06-03T00:51:20.990ZDEBGnexus (ServiceBalancer): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Rack { parent: Fleet, key: d6831ba0-55f0-4191-8dc2-ea15e55ab0ae, lookup_type: ById(d6831ba0-55f0-4191-8dc2-ea15e55ab0ae) } result = Ok(())
9632025-06-03T00:51:20.990ZINFOnexus: Still waiting for rack initialization: Rack { identity: RackIdentity { id: d6831ba0-55f0-4191-8dc2-ea15e55ab0ae, time_created: 2025-06-03T00:51:10.288786Z, time_modified: 2025-06-03T00:51:10.288786Z }, initialized: false, tuf_base_url: None, rack_subnet: Some(V6(Ipv6Network { addr: fd00:1122:3344:100::, prefix: 56 })) } file = nexus/src/app/rack.rs:775
9642025-06-03T00:51:22.991ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
9652025-06-03T00:51:22.992ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
9662025-06-03T00:51:22.994ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
9672025-06-03T00:51:22.994ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
9682025-06-03T00:51:22.996ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9692025-06-03T00:51:22.998ZDEBGnexus (ServiceBalancer): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Rack { parent: Fleet, key: d6831ba0-55f0-4191-8dc2-ea15e55ab0ae, lookup_type: ById(d6831ba0-55f0-4191-8dc2-ea15e55ab0ae) } result = Ok(())
9702025-06-03T00:51:22.998ZINFOnexus: Still waiting for rack initialization: Rack { identity: RackIdentity { id: d6831ba0-55f0-4191-8dc2-ea15e55ab0ae, time_created: 2025-06-03T00:51:10.288786Z, time_modified: 2025-06-03T00:51:10.288786Z }, initialized: false, tuf_base_url: None, rack_subnet: Some(V6(Ipv6Network { addr: fd00:1122:3344:100::, prefix: 56 })) } file = nexus/src/app/rack.rs:775
9712025-06-03T00:51:25.000ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
9722025-06-03T00:51:25.000ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
9732025-06-03T00:51:25.002ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
9742025-06-03T00:51:25.002ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
9752025-06-03T00:51:25.004ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9762025-06-03T00:51:25.006ZDEBGnexus (ServiceBalancer): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Rack { parent: Fleet, key: d6831ba0-55f0-4191-8dc2-ea15e55ab0ae, lookup_type: ById(d6831ba0-55f0-4191-8dc2-ea15e55ab0ae) } result = Ok(())
9772025-06-03T00:51:25.006ZINFOnexus: Still waiting for rack initialization: Rack { identity: RackIdentity { id: d6831ba0-55f0-4191-8dc2-ea15e55ab0ae, time_created: 2025-06-03T00:51:10.288786Z, time_modified: 2025-06-03T00:51:10.288786Z }, initialized: false, tuf_base_url: None, rack_subnet: Some(V6(Ipv6Network { addr: fd00:1122:3344:100::, prefix: 56 })) } file = nexus/src/app/rack.rs:775
9782025-06-03T00:51:27.008ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
9792025-06-03T00:51:27.008ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
9802025-06-03T00:51:27.010ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
9812025-06-03T00:51:27.010ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
9822025-06-03T00:51:27.012ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9832025-06-03T00:51:27.014ZDEBGnexus (ServiceBalancer): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Rack { parent: Fleet, key: d6831ba0-55f0-4191-8dc2-ea15e55ab0ae, lookup_type: ById(d6831ba0-55f0-4191-8dc2-ea15e55ab0ae) } result = Ok(())
9842025-06-03T00:51:27.014ZINFOnexus: Still waiting for rack initialization: Rack { identity: RackIdentity { id: d6831ba0-55f0-4191-8dc2-ea15e55ab0ae, time_created: 2025-06-03T00:51:10.288786Z, time_modified: 2025-06-03T00:51:10.288786Z }, initialized: false, tuf_base_url: None, rack_subnet: Some(V6(Ipv6Network { addr: fd00:1122:3344:100::, prefix: 56 })) } file = nexus/src/app/rack.rs:775
9852025-06-03T00:51:29.016ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
9862025-06-03T00:51:29.016ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
9872025-06-03T00:51:29.018ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
9882025-06-03T00:51:29.018ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
9892025-06-03T00:51:29.020ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9902025-06-03T00:51:29.023ZDEBGnexus (ServiceBalancer): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Rack { parent: Fleet, key: d6831ba0-55f0-4191-8dc2-ea15e55ab0ae, lookup_type: ById(d6831ba0-55f0-4191-8dc2-ea15e55ab0ae) } result = Ok(())
9912025-06-03T00:51:29.023ZINFOnexus: Still waiting for rack initialization: Rack { identity: RackIdentity { id: d6831ba0-55f0-4191-8dc2-ea15e55ab0ae, time_created: 2025-06-03T00:51:10.288786Z, time_modified: 2025-06-03T00:51:10.288786Z }, initialized: false, tuf_base_url: None, rack_subnet: Some(V6(Ipv6Network { addr: fd00:1122:3344:100::, prefix: 56 })) } file = nexus/src/app/rack.rs:775
9922025-06-03T00:51:30.508ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 3 reason = Timeout
9932025-06-03T00:51:30.508ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
9942025-06-03T00:51:30.508ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 3 reason = Timeout
9952025-06-03T00:51:30.508ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9962025-06-03T00:51:30.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
9972025-06-03T00:51:30.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9982025-06-03T00:51:30.510ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
9992025-06-03T00:51:30.510ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10002025-06-03T00:51:30.510ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
10012025-06-03T00:51:30.513ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
10022025-06-03T00:51:30.513ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
10032025-06-03T00:51:30.513ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
10042025-06-03T00:51:30.515ZWARNnexus (BackgroundTasks): failed to read target blueprint background_task = blueprint_loader error = Internal Error: no target blueprint set file = nexus/src/app/background/tasks/blueprint_load.rs:69
10052025-06-03T00:51:30.515ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 6.992427ms iteration = 3 status = {"error":"failed to read target blueprint: Internal Error: no target blueprint set"}
10062025-06-03T00:51:30.543ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 35.584955ms iteration = 3 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
10072025-06-03T00:51:31.023ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
10082025-06-03T00:51:31.024ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
10092025-06-03T00:51:31.026ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
10102025-06-03T00:51:31.026ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
10112025-06-03T00:51:31.028ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10122025-06-03T00:51:31.030ZDEBGnexus (ServiceBalancer): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Rack { parent: Fleet, key: d6831ba0-55f0-4191-8dc2-ea15e55ab0ae, lookup_type: ById(d6831ba0-55f0-4191-8dc2-ea15e55ab0ae) } result = Ok(())
10132025-06-03T00:51:31.030ZINFOnexus: Still waiting for rack initialization: Rack { identity: RackIdentity { id: d6831ba0-55f0-4191-8dc2-ea15e55ab0ae, time_created: 2025-06-03T00:51:10.288786Z, time_modified: 2025-06-03T00:51:10.288786Z }, initialized: false, tuf_base_url: None, rack_subnet: Some(V6(Ipv6Network { addr: fd00:1122:3344:100::, prefix: 56 })) } file = nexus/src/app/rack.rs:775
10142025-06-03T00:51:33.032ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
10152025-06-03T00:51:33.033ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
10162025-06-03T00:51:33.033ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
10172025-06-03T00:51:33.034ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
10182025-06-03T00:51:33.035ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10192025-06-03T00:51:33.037ZDEBGnexus (ServiceBalancer): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Rack { parent: Fleet, key: d6831ba0-55f0-4191-8dc2-ea15e55ab0ae, lookup_type: ById(d6831ba0-55f0-4191-8dc2-ea15e55ab0ae) } result = Ok(())
10202025-06-03T00:51:33.037ZINFOnexus: Still waiting for rack initialization: Rack { identity: RackIdentity { id: d6831ba0-55f0-4191-8dc2-ea15e55ab0ae, time_created: 2025-06-03T00:51:10.288786Z, time_modified: 2025-06-03T00:51:10.288786Z }, initialized: false, tuf_base_url: None, rack_subnet: Some(V6(Ipv6Network { addr: fd00:1122:3344:100::, prefix: 56 })) } file = nexus/src/app/rack.rs:775
10212025-06-03T00:51:35.039ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
10222025-06-03T00:51:35.039ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
10232025-06-03T00:51:35.041ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
10242025-06-03T00:51:35.041ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
10252025-06-03T00:51:35.043ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10262025-06-03T00:51:35.046ZDEBGnexus (ServiceBalancer): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Rack { parent: Fleet, key: d6831ba0-55f0-4191-8dc2-ea15e55ab0ae, lookup_type: ById(d6831ba0-55f0-4191-8dc2-ea15e55ab0ae) } result = Ok(())
10272025-06-03T00:51:35.046ZINFOnexus: Still waiting for rack initialization: Rack { identity: RackIdentity { id: d6831ba0-55f0-4191-8dc2-ea15e55ab0ae, time_created: 2025-06-03T00:51:10.288786Z, time_modified: 2025-06-03T00:51:10.288786Z }, initialized: false, tuf_base_url: None, rack_subnet: Some(V6(Ipv6Network { addr: fd00:1122:3344:100::, prefix: 56 })) } file = nexus/src/app/rack.rs:775
10282025-06-03T00:51:37.046ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
10292025-06-03T00:51:37.047ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
10302025-06-03T00:51:37.049ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
10312025-06-03T00:51:37.049ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
10322025-06-03T00:51:37.050ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10332025-06-03T00:51:37.053ZDEBGnexus (ServiceBalancer): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Rack { parent: Fleet, key: d6831ba0-55f0-4191-8dc2-ea15e55ab0ae, lookup_type: ById(d6831ba0-55f0-4191-8dc2-ea15e55ab0ae) } result = Ok(())
10342025-06-03T00:51:37.053ZINFOnexus: Still waiting for rack initialization: Rack { identity: RackIdentity { id: d6831ba0-55f0-4191-8dc2-ea15e55ab0ae, time_created: 2025-06-03T00:51:10.288786Z, time_modified: 2025-06-03T00:51:10.288786Z }, initialized: false, tuf_base_url: None, rack_subnet: Some(V6(Ipv6Network { addr: fd00:1122:3344:100::, prefix: 56 })) } file = nexus/src/app/rack.rs:775
10352025-06-03T00:51:39.053ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
10362025-06-03T00:51:39.053ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
10372025-06-03T00:51:39.055ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
10382025-06-03T00:51:39.055ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
10392025-06-03T00:51:39.057ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10402025-06-03T00:51:39.059ZDEBGnexus (ServiceBalancer): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Rack { parent: Fleet, key: d6831ba0-55f0-4191-8dc2-ea15e55ab0ae, lookup_type: ById(d6831ba0-55f0-4191-8dc2-ea15e55ab0ae) } result = Ok(())
10412025-06-03T00:51:39.059ZINFOnexus: Rack initialized file = nexus/src/app/rack.rs:772
10422025-06-03T00:51:39.063ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
10432025-06-03T00:51:39.063ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
10442025-06-03T00:51:39.065ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
10452025-06-03T00:51:39.065ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
10462025-06-03T00:51:39.067ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
10472025-06-03T00:51:39.067ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
10482025-06-03T00:51:39.071ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
10492025-06-03T00:51:39.071ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
10502025-06-03T00:51:39.072ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
10512025-06-03T00:51:39.073ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
10522025-06-03T00:51:39.074ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
10532025-06-03T00:51:39.074ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
10542025-06-03T00:51:39.075ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10552025-06-03T00:51:39.078ZDEBGnexus (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
10562025-06-03T00:51:39.078ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
10572025-06-03T00:51:39.079ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
10582025-06-03T00:51:39.080ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
10592025-06-03T00:51:39.080ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
10602025-06-03T00:51:39.081ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
10612025-06-03T00:51:39.081ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
10622025-06-03T00:51:39.082ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10632025-06-03T00:51:39.086ZDEBGnexus (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
10642025-06-03T00:51:39.086ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
10652025-06-03T00:51:39.086ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
10662025-06-03T00:51:39.089ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
10672025-06-03T00:51:39.089ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
10682025-06-03T00:51:39.090ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
10692025-06-03T00:51:39.090ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
10702025-06-03T00:51:39.092ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
10712025-06-03T00:51:39.092ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
10722025-06-03T00:51:39.093ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
10732025-06-03T00:51:39.093ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
10742025-06-03T00:51:39.094ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
10752025-06-03T00:51:39.095ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
10762025-06-03T00:51:39.096ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
10772025-06-03T00:51:39.096ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
10782025-06-03T00:51:39.097ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10792025-06-03T00:51:39.101ZDEBGnexus (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
10802025-06-03T00:51:39.102ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
10812025-06-03T00:51:39.102ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
10822025-06-03T00:51:39.103ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
10832025-06-03T00:51:39.103ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
10842025-06-03T00:51:39.105ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
10852025-06-03T00:51:39.105ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
10862025-06-03T00:51:39.107ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
10872025-06-03T00:51:39.108ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
10882025-06-03T00:51:39.110ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
10892025-06-03T00:51:39.110ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
10902025-06-03T00:51:39.111ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
10912025-06-03T00:51:39.111ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
10922025-06-03T00:51:39.113ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
10932025-06-03T00:51:39.113ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
10942025-06-03T00:51:39.114ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10952025-06-03T00:51:39.118ZDEBGnexus (InternalApi): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
10962025-06-03T00:51:39.118ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
10972025-06-03T00:51:39.118ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
10982025-06-03T00:51:39.119ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
10992025-06-03T00:51:39.119ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
11002025-06-03T00:51:39.121ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
11012025-06-03T00:51:39.121ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
11022025-06-03T00:51:39.122ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11032025-06-03T00:51:39.126ZDEBGnexus (InternalApi): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
11042025-06-03T00:51:39.126ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
11052025-06-03T00:51:39.126ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
11062025-06-03T00:51:39.135ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
11072025-06-03T00:51:39.135ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
11082025-06-03T00:51:39.137ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
11092025-06-03T00:51:39.137ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
11102025-06-03T00:51:39.229ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
11112025-06-03T00:51:39.230ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
11122025-06-03T00:51:39.232ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
11132025-06-03T00:51:39.232ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
11142025-06-03T00:51:39.234ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
11152025-06-03T00:51:39.234ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
11162025-06-03T00:51:39.236ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
11172025-06-03T00:51:39.236ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
11182025-06-03T00:51:39.237ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
11192025-06-03T00:51:39.237ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
11202025-06-03T00:51:39.243ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11212025-06-03T00:51:39.247ZDEBGnexus (InternalApi): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
11222025-06-03T00:51:39.247ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
11232025-06-03T00:51:39.247ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
11242025-06-03T00:51:39.249ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
11252025-06-03T00:51:39.249ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
11262025-06-03T00:51:39.250ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
11272025-06-03T00:51:39.250ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
11282025-06-03T00:51:39.252ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11292025-06-03T00:51:39.255ZDEBGnexus (InternalApi): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
11302025-06-03T00:51:39.255ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
11312025-06-03T00:51:39.255ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
11322025-06-03T00:51:39.262ZDEBGnexus (InternalApi): resolved names for firewall rules instance_interfaces = {} subnet_interfaces = {Name("nexus"): [NetworkInterface { id: 68a89d5e-a312-4992-a0a9-921a41d0d3dd, kind: Service { id: 185a1d3b-437a-4bdb-baa0-fc5d8728a3a8 }, name: Name("nexus-185a1d3b-437a-4bdb-baa0-fc5d8728a3a8"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 199, 152])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 68ff820c-70a6-4b17-95d5-d3c31f7b4c51, kind: Service { id: fe12ffc1-f2ac-4bf3-9672-fc185f82c59b }, name: Name("nexus-fe12ffc1-f2ac-4bf3-9672-fc185f82c59b"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 216, 245])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: ece06ff8-68bc-4632-b601-b8b26de7e08d, kind: Service { id: a2861392-5fd2-462b-84e5-6f36c716263a }, name: Name("nexus-a2861392-5fd2-462b-84e5-6f36c716263a"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 243, 2])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], Name("external-dns"): [NetworkInterface { id: 3aad1b77-42d7-48c8-9121-0ef5c4effed9, kind: Service { id: aa124775-31de-4648-981a-4af500268b66 }, name: Name("external-dns-aa124775-31de-4648-981a-4af500268b66"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 180, 88])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: a9971f4d-e41a-4de5-9ec6-26e6471eb4a4, kind: Service { id: 9feb16c9-ff60-43e8-917b-965124f46e0d }, name: Name("external-dns-9feb16c9-ff60-43e8-917b-965124f46e0d"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 190, 11])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }]} subnet_networks = {Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })], Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })]} vpc_interfaces = {}
11332025-06-03T00:51:39.262ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
11342025-06-03T00:51:39.262ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
11352025-06-03T00:51:39.264ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11362025-06-03T00:51:39.265ZDEBGnexus (InternalApi): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet result = Ok(())
11372025-06-03T00:51:39.265ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
11382025-06-03T00:51:39.265ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
11392025-06-03T00:51:39.267ZDEBGnexus (InternalApi): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified.
11402025-06-03T00:51:39.273ZDEBGnexus (InternalApi): resolved firewall rules for sled agents sled_agent_rules = [ResolvedVpcFirewallRule { action: Allow, direction: Inbound, filter_hosts: None, filter_ports: Some([L4PortRange("53")]), filter_protocols: Some([Udp]), priority: 65534, status: Enabled, targets: [NetworkInterface { id: 3aad1b77-42d7-48c8-9121-0ef5c4effed9, kind: Service { id: aa124775-31de-4648-981a-4af500268b66 }, name: Name("external-dns-aa124775-31de-4648-981a-4af500268b66"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 180, 88])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: a9971f4d-e41a-4de5-9ec6-26e6471eb4a4, kind: Service { id: 9feb16c9-ff60-43e8-917b-965124f46e0d }, name: Name("external-dns-9feb16c9-ff60-43e8-917b-965124f46e0d"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 190, 11])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }] }, ResolvedVpcFirewallRule { action: Allow, direction: Inbound, filter_hosts: None, filter_ports: Some([L4PortRange("80"), L4PortRange("443")]), filter_protocols: Some([Tcp]), priority: 65534, status: Enabled, targets: [NetworkInterface { id: 68a89d5e-a312-4992-a0a9-921a41d0d3dd, kind: Service { id: 185a1d3b-437a-4bdb-baa0-fc5d8728a3a8 }, name: Name("nexus-185a1d3b-437a-4bdb-baa0-fc5d8728a3a8"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 199, 152])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 68ff820c-70a6-4b17-95d5-d3c31f7b4c51, kind: Service { id: fe12ffc1-f2ac-4bf3-9672-fc185f82c59b }, name: Name("nexus-fe12ffc1-f2ac-4bf3-9672-fc185f82c59b"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 216, 245])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: ece06ff8-68bc-4632-b601-b8b26de7e08d, kind: Service { id: a2861392-5fd2-462b-84e5-6f36c716263a }, name: Name("nexus-a2861392-5fd2-462b-84e5-6f36c716263a"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 243, 2])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }] }]
11412025-06-03T00:51:39.273ZDEBGnexus (InternalApi): resolved 2 rules for sleds
11422025-06-03T00:51:39.331ZDEBGnexus (InternalApi): resolved sleds for vpc oxide-services vpc_to_sled = [Sled { identity: SledIdentity { id: afe89ed7-19b7-479b-81eb-05c988ebc66d, time_created: 2025-06-03T00:51:37.177791Z, time_modified: 2025-06-03T00:51:37.177791Z }, time_deleted: None, rcgen: Generation(Generation(19)), rack_id: d6831ba0-55f0-4191-8dc2-ea15e55ab0ae, is_scrimlet: true, serial_number: "sock", part_number: "i86pc", revision: SqlU32(0), usable_hardware_threads: SqlU32(48), usable_physical_ram: ByteCount(ByteCount(102952026112)), reservoir_size: ByteCount(ByteCount(56094621696)), ip: fd00:1122:3344:101::1, port: SqlU16(12345), last_used_address: fd00:1122:3344:101::ffff, policy: InService, state: Active, sled_agent_gen: Generation(Generation(1)), repo_depot_port: SqlU16(12348) }]
11432025-06-03T00:51:39.331ZDEBGnexus (InternalApi): sending firewall rules to sled agents
11442025-06-03T00:51:39.363ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
11452025-06-03T00:51:39.364ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
11462025-06-03T00:51:39.366ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
11472025-06-03T00:51:39.366ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
11482025-06-03T00:51:39.368ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11492025-06-03T00:51:39.370ZDEBGnexus (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Sled { parent: Fleet, key: afe89ed7-19b7-479b-81eb-05c988ebc66d, lookup_type: ById(afe89ed7-19b7-479b-81eb-05c988ebc66d) } result = Ok(())
11502025-06-03T00:51:39.371ZDEBGnexus (InternalApi): client request SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules
11512025-06-03T00:51:39.371ZDEBGnexus (InternalApi): client response SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules", status: 204, headers: {"x-request-id": "513169c3-e77e-4663-921f-e941e91eda21", "date": "Tue, 03 Jun 2025 00:51:39 GMT"} })
11522025-06-03T00:51:39.372ZINFOnexus (InternalApi): updated firewall rules on 1 sleds file = nexus/networking/src/firewall_rules.rs:495
11532025-06-03T00:51:39.372ZINFOnexus: plumbed initial IP allowlist file = nexus/src/app/allow_list.rs:161
11542025-06-03T00:51:40.508ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 4 reason = Timeout
11552025-06-03T00:51:40.508ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
11562025-06-03T00:51:40.508ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 2 reason = Timeout
11572025-06-03T00:51:40.508ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 2 reason = Timeout
11582025-06-03T00:51:40.508ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
11592025-06-03T00:51:40.508ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
11602025-06-03T00:51:40.508ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 2 reason = Timeout
11612025-06-03T00:51:40.508ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 2 reason = Timeout
11622025-06-03T00:51:40.508ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
11632025-06-03T00:51:40.508ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
11642025-06-03T00:51:40.508ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
11652025-06-03T00:51:40.508ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 2 reason = Timeout
11662025-06-03T00:51:40.508ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 2 reason = Timeout
11672025-06-03T00:51:40.508ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
11682025-06-03T00:51:40.508ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 3 reason = Timeout
11692025-06-03T00:51:40.508ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
11702025-06-03T00:51:40.508ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
11712025-06-03T00:51:40.508ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 2 reason = Timeout
11722025-06-03T00:51:40.508ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 2 reason = Timeout
11732025-06-03T00:51:40.508ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
11742025-06-03T00:51:40.508ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 2 reason = Timeout
11752025-06-03T00:51:40.508ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
11762025-06-03T00:51:40.508ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
11772025-06-03T00:51:40.508ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
11782025-06-03T00:51:40.508ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 4 reason = Timeout
11792025-06-03T00:51:40.508ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
11802025-06-03T00:51:40.508ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 2 reason = Timeout
11812025-06-03T00:51:40.508ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
11822025-06-03T00:51:40.508ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 2 reason = Timeout
11832025-06-03T00:51:40.508ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
11842025-06-03T00:51:40.508ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 2 reason = Timeout
11852025-06-03T00:51:40.508ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
11862025-06-03T00:51:40.508ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 2 reason = Timeout
11872025-06-03T00:51:40.508ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
11882025-06-03T00:51:40.508ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 2 reason = Timeout
11892025-06-03T00:51:40.508ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
11902025-06-03T00:51:40.508ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 2 reason = Timeout
11912025-06-03T00:51:40.508ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 2 reason = Timeout
11922025-06-03T00:51:40.508ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
11932025-06-03T00:51:40.509ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
11942025-06-03T00:51:40.509ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
11952025-06-03T00:51:40.509ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
11962025-06-03T00:51:40.509ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
11972025-06-03T00:51:40.509ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
11982025-06-03T00:51:40.509ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
11992025-06-03T00:51:40.509ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
12002025-06-03T00:51:40.509ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
12012025-06-03T00:51:40.509ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
12022025-06-03T00:51:40.509ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
12032025-06-03T00:51:40.509ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
12042025-06-03T00:51:40.509ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
12052025-06-03T00:51:40.509ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
12062025-06-03T00:51:40.509ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
12072025-06-03T00:51:40.509ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
12082025-06-03T00:51:40.509ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
12092025-06-03T00:51:40.509ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
12102025-06-03T00:51:40.509ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
12112025-06-03T00:51:40.511ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.144648ms iteration = 2 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
12122025-06-03T00:51:40.512ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 4.491629ms iteration = 2 status = {"errors":[],"garbage_collect_requested":[]}
12132025-06-03T00:51:40.513ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12142025-06-03T00:51:40.513ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12152025-06-03T00:51:40.514ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12162025-06-03T00:51:40.515ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
12172025-06-03T00:51:40.515ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
12182025-06-03T00:51:40.516ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12192025-06-03T00:51:40.516ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
12202025-06-03T00:51:40.516ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
12212025-06-03T00:51:40.516ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
12222025-06-03T00:51:40.518ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
12232025-06-03T00:51:40.518ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
12242025-06-03T00:51:40.518ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
12252025-06-03T00:51:40.518ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
12262025-06-03T00:51:40.518ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
12272025-06-03T00:51:40.518ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
12282025-06-03T00:51:40.519ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
12292025-06-03T00:51:40.519ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
12302025-06-03T00:51:40.519ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
12312025-06-03T00:51:40.521ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12322025-06-03T00:51:40.524ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
12332025-06-03T00:51:40.524ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
12342025-06-03T00:51:40.524ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
12352025-06-03T00:51:40.529ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
12362025-06-03T00:51:40.530ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
12372025-06-03T00:51:40.530ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
12382025-06-03T00:51:40.530ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
12392025-06-03T00:51:40.530ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
12402025-06-03T00:51:40.531ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
12412025-06-03T00:51:40.533ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12422025-06-03T00:51:40.536ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
12432025-06-03T00:51:40.536ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
12442025-06-03T00:51:40.536ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
12452025-06-03T00:51:40.544ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-afe89ed7-19b7-479b-81eb-05c988ebc66d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-afe89ed7-19b7-479b-81eb-05c988ebc66d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 2427, tv_nsec: 766068458 } })
12462025-06-03T00:51:40.544ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1179
12472025-06-03T00:51:40.544ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-afe89ed7-19b7-479b-81eb-05c988ebc66d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-afe89ed7-19b7-479b-81eb-05c988ebc66d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 2427, tv_nsec: 766622125 } })
12482025-06-03T00:51:40.545ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1179
12492025-06-03T00:51:40.564ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
12502025-06-03T00:51:40.564ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
12512025-06-03T00:51:40.564ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
12522025-06-03T00:51:40.564ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
12532025-06-03T00:51:40.581ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
12542025-06-03T00:51:40.581ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
12552025-06-03T00:51:40.583ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12562025-06-03T00:51:40.584ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 76.161816ms iteration = 2 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
12572025-06-03T00:51:40.586ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
12582025-06-03T00:51:40.586ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
12592025-06-03T00:51:40.586ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
12602025-06-03T00:51:40.588ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12612025-06-03T00:51:40.588ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
12622025-06-03T00:51:40.589ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
12632025-06-03T00:51:40.591ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 82.534963ms iteration = 2 status = {"errors":[],"requests_created_ok":[]}
12642025-06-03T00:51:40.591ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
12652025-06-03T00:51:40.591ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
12662025-06-03T00:51:40.591ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
12672025-06-03T00:51:40.592ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 83.633987ms iteration = 4 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
12682025-06-03T00:51:40.593ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12692025-06-03T00:51:40.593ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12702025-06-03T00:51:40.594ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
12712025-06-03T00:51:40.594ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
12722025-06-03T00:51:40.594ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
12732025-06-03T00:51:40.594ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
12742025-06-03T00:51:40.596ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
12752025-06-03T00:51:40.596ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
12762025-06-03T00:51:40.596ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
12772025-06-03T00:51:40.597ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 4ec93699-353f-4b9e-ae06-25355fdb069e, lookup_type: ById(4ec93699-353f-4b9e-ae06-25355fdb069e) } result = Ok(())
12782025-06-03T00:51:40.597ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
12792025-06-03T00:51:40.598ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
12802025-06-03T00:51:40.598ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 90.025485ms iteration = 2 status = {"errors":[],"finish_invoked_ok":[]}
12812025-06-03T00:51:40.600ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-afe89ed7-19b7-479b-81eb-05c988ebc66d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-afe89ed7-19b7-479b-81eb-05c988ebc66d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 2427, tv_nsec: 822522480 } })
12822025-06-03T00:51:40.601ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1179 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae
12832025-06-03T00:51:40.603ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
12842025-06-03T00:51:40.603ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
12852025-06-03T00:51:40.605ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12862025-06-03T00:51:40.607ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
12872025-06-03T00:51:40.607ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
12882025-06-03T00:51:40.608ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
12892025-06-03T00:51:40.608ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:547 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
12902025-06-03T00:51:40.608ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 99.86801ms iteration = 2 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
12912025-06-03T00:51:40.614ZINFOnexus (BackgroundTasks): found new target blueprint (first find) background_task = blueprint_loader file = nexus/src/app/background/tasks/blueprint_load.rs:88 target_id = 4ec93699-353f-4b9e-ae06-25355fdb069e time_created = 2025-06-03 00:51:07.059326 UTC
12922025-06-03T00:51:40.614ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 2 reason = Dependency
12932025-06-03T00:51:40.615ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 106.346072ms iteration = 4 status = {"enabled":false,"status":"first target blueprint","target_id":"4ec93699-353f-4b9e-ae06-25355fdb069e","time_created":"2025-06-03T00:51:07.059326Z","time_found":"2025-06-03T00:51:40.614980804Z"}
12942025-06-03T00:51:40.615ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:85 reason = blueprint disabled target_id = 4ec93699-353f-4b9e-ae06-25355fdb069e
12952025-06-03T00:51:40.615ZDEBGnexus (BackgroundTasks): activating background_task = crdb_node_id_collector iteration = 2 reason = Dependency
12962025-06-03T00:51:40.615ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 21.191\u{b5}s iteration = 2 status = {"enabled":false,"target_id":"4ec93699-353f-4b9e-ae06-25355fdb069e"}
12972025-06-03T00:51:40.615ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
12982025-06-03T00:51:40.615ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
12992025-06-03T00:51:40.615ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
13002025-06-03T00:51:40.615ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
13012025-06-03T00:51:40.615ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
13022025-06-03T00:51:40.615ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
13032025-06-03T00:51:40.615ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
13042025-06-03T00:51:40.616ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
13052025-06-03T00:51:40.616ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
13062025-06-03T00:51:40.616ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
13072025-06-03T00:51:40.618ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13082025-06-03T00:51:40.620ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
13092025-06-03T00:51:40.622ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
13102025-06-03T00:51:40.622ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
13112025-06-03T00:51:40.623ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13122025-06-03T00:51:40.625ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
13132025-06-03T00:51:40.625ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
13142025-06-03T00:51:40.625ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
13152025-06-03T00:51:40.625ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13162025-06-03T00:51:40.626ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
13172025-06-03T00:51:40.626ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
13182025-06-03T00:51:40.626ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
13192025-06-03T00:51:40.627ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
13202025-06-03T00:51:40.628ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
13212025-06-03T00:51:40.628ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
13222025-06-03T00:51:40.628ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
13232025-06-03T00:51:40.628ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
13242025-06-03T00:51:40.628ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
13252025-06-03T00:51:40.632ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
13262025-06-03T00:51:40.633ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
13272025-06-03T00:51:40.640ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13282025-06-03T00:51:40.644ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: afe89ed7-19b7-479b-81eb-05c988ebc66d, lookup_type: ById(afe89ed7-19b7-479b-81eb-05c988ebc66d) } result = Ok(())
13292025-06-03T00:51:40.644ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
13302025-06-03T00:51:40.644ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
13312025-06-03T00:51:40.650ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
13322025-06-03T00:51:40.650ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
13332025-06-03T00:51:40.651ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 142.813794ms iteration = 2 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
13342025-06-03T00:51:40.655ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
13352025-06-03T00:51:40.656ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
13362025-06-03T00:51:40.664ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:372
13372025-06-03T00:51:40.664ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 156.116207ms iteration = 2 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
13382025-06-03T00:51:40.672ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
13392025-06-03T00:51:40.672ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
13402025-06-03T00:51:40.672ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 164.176708ms iteration = 2 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
13412025-06-03T00:51:40.690ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 181.293109ms iteration = 2 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
13422025-06-03T00:51:40.697ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 188.474452ms iteration = 3 status = {"physical_disks_added":0}
13432025-06-03T00:51:40.731ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1187 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae zone_address = fd00:1122:3344:101::2
13442025-06-03T00:51:40.731ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
13452025-06-03T00:51:40.731ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "5aaeb2d7-b066-4764-a9e1-5cb9349fe203", "content-length": "26", "date": "Tue, 03 Jun 2025 00:51:40 GMT"} })
13462025-06-03T00:51:40.731ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1190 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
13472025-06-03T00:51:40.733ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1224 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae
13482025-06-03T00:51:40.739ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
13492025-06-03T00:51:40.739ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1187 zone_address = fd00:1122:3344:101::2
13502025-06-03T00:51:40.739ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
13512025-06-03T00:51:40.739ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
13522025-06-03T00:51:40.739ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "331a7074-3426-41ad-bd71-4786dc6bc8f7", "content-length": "26", "date": "Tue, 03 Jun 2025 00:51:40 GMT"} })
13532025-06-03T00:51:40.739ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1190 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
13542025-06-03T00:51:40.740ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1224 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
13552025-06-03T00:51:40.744ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1187 zone_address = fd00:1122:3344:101::2
13562025-06-03T00:51:40.744ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
13572025-06-03T00:51:40.745ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "9ba01ff0-eddf-45e3-8046-4d9556de79c2", "content-length": "26", "date": "Tue, 03 Jun 2025 00:51:40 GMT"} })
13582025-06-03T00:51:40.745ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1190 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
13592025-06-03T00:51:40.746ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1224 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
13602025-06-03T00:51:40.790ZDEBGnexus (BackgroundTasks): client request background_task = crdb_node_id_collector body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/node/id
13612025-06-03T00:51:40.849ZINFOnexus (BackgroundTasks): VPC route manager sled afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
13622025-06-03T00:51:40.849ZDEBGnexus (BackgroundTasks): client request SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
13632025-06-03T00:51:40.850ZDEBGnexus (BackgroundTasks): client response SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "fd4fb1a6-5b79-4a41-b3b7-7c6e6d9068b5", "content-length": "312", "date": "Tue, 03 Jun 2025 00:51:40 GMT"} })
13642025-06-03T00:51:40.852ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
13652025-06-03T00:51:40.852ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
13662025-06-03T00:51:40.891ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
13672025-06-03T00:51:40.892ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
13682025-06-03T00:51:40.929ZDEBGnexus (BackgroundTasks): client request background_task = crdb_node_id_collector body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/node/id
13692025-06-03T00:51:40.984ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 476.31908ms iteration = 2 status = {}
13702025-06-03T00:51:41.011ZDEBGnexus (BackgroundTasks): client request background_task = crdb_node_id_collector body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/node/id
13712025-06-03T00:51:41.011ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13722025-06-03T00:51:41.014ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
13732025-06-03T00:51:41.014ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
13742025-06-03T00:51:41.014ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
13752025-06-03T00:51:41.014ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13762025-06-03T00:51:41.014ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13772025-06-03T00:51:41.015ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
13782025-06-03T00:51:41.015ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
13792025-06-03T00:51:41.016ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 507.971743ms iteration = 2 status = null
13802025-06-03T00:51:41.016ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
13812025-06-03T00:51:41.016ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
13822025-06-03T00:51:41.017ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
13832025-06-03T00:51:41.017ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
13842025-06-03T00:51:41.017ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
13852025-06-03T00:51:41.017ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
13862025-06-03T00:51:41.020ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
13872025-06-03T00:51:41.021ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
13882025-06-03T00:51:41.063ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13892025-06-03T00:51:41.065ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
13902025-06-03T00:51:41.065ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
13912025-06-03T00:51:41.065ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
13922025-06-03T00:51:41.082ZDEBGnexus (BackgroundTasks): client request background_task = crdb_node_id_collector body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/node/id
13932025-06-03T00:51:41.085ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
13942025-06-03T00:51:41.086ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
13952025-06-03T00:51:41.088ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
13962025-06-03T00:51:41.088ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
13972025-06-03T00:51:41.089ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
13982025-06-03T00:51:41.089ZDEBGnexus (BackgroundTasks): client request SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
13992025-06-03T00:51:41.090ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
14002025-06-03T00:51:41.091ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
14012025-06-03T00:51:41.093ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
14022025-06-03T00:51:41.093ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
14032025-06-03T00:51:41.121ZDEBGnexus (BackgroundTasks): client request background_task = crdb_node_id_collector body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/node/id
14042025-06-03T00:51:41.123ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
14052025-06-03T00:51:41.124ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
14062025-06-03T00:51:41.124ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: ec5e33f9-c697-46e2-a93e-a745336bf9dc, rack_id: d6831ba0-55f0-4191-8dc2-ea15e55ab0ae, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(9378a284-0ee9-48e2-80a0-3bcaadb1c4e2) } rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae switch_location = Switch0
14072025-06-03T00:51:41.124ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
14082025-06-03T00:51:41.125ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
14092025-06-03T00:51:41.129ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
14102025-06-03T00:51:41.129ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
14112025-06-03T00:51:41.132ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
14122025-06-03T00:51:41.132ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
14132025-06-03T00:51:41.134ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
14142025-06-03T00:51:41.135ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
14152025-06-03T00:51:41.138ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
14162025-06-03T00:51:41.138ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
14172025-06-03T00:51:41.140ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
14182025-06-03T00:51:41.141ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
14192025-06-03T00:51:41.144ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-afe89ed7-19b7-479b-81eb-05c988ebc66d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-afe89ed7-19b7-479b-81eb-05c988ebc66d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 2428, tv_nsec: 365934914 } })
14202025-06-03T00:51:41.144ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = service_zone_nat_tracker file = nexus/src/app/mod.rs:1179
14212025-06-03T00:51:41.194ZDEBGnexus (BackgroundTasks): client response SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "a3d1f84d-7ae9-4bf7-8331-36f421df8e09", "date": "Tue, 03 Jun 2025 00:51:40 GMT"} })
14222025-06-03T00:51:41.195ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
14232025-06-03T00:51:41.195ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
14242025-06-03T00:51:41.197ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
14252025-06-03T00:51:41.198ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
14262025-06-03T00:51:41.201ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
14272025-06-03T00:51:41.202ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
14282025-06-03T00:51:41.206ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = service_zone_nat_tracker file = nexus/src/app/mod.rs:1187 zone_address = fd00:1122:3344:101::2
14292025-06-03T00:51:41.206ZDEBGnexus (MgsClient): client request background_task = service_zone_nat_tracker body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
14302025-06-03T00:51:41.207ZDEBGnexus (MgsClient): client response background_task = service_zone_nat_tracker result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "b7d8b9f5-7a2b-4407-9f14-10ae0f104002", "content-length": "26", "date": "Tue, 03 Jun 2025 00:51:40 GMT"} })
14312025-06-03T00:51:41.207ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = service_zone_nat_tracker file = nexus/src/app/mod.rs:1190 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
14322025-06-03T00:51:41.208ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = service_zone_nat_tracker file = nexus/src/app/mod.rs:1224 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
14332025-06-03T00:51:41.218ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1832 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
14342025-06-03T00:51:41.218ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1859 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 9378a284-0ee9-48e2-80a0-3bcaadb1c4e2
14352025-06-03T00:51:41.219ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
14362025-06-03T00:51:41.219ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
14372025-06-03T00:51:41.219ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
14382025-06-03T00:51:41.219ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
14392025-06-03T00:51:41.219ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
14402025-06-03T00:51:41.219ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2006 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
14412025-06-03T00:51:41.219ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:408 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
14422025-06-03T00:51:41.219ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2042 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
14432025-06-03T00:51:41.219ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae
14442025-06-03T00:51:41.219ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
14452025-06-03T00:51:41.220ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
14462025-06-03T00:51:41.228ZDEBGnexus (BackgroundTasks): client request SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
14472025-06-03T00:51:41.229ZDEBGnexus (BackgroundTasks): client response SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "cbfdba90-bc37-436d-ae2b-b485964927fb", "content-length": "691", "date": "Tue, 03 Jun 2025 00:51:41 GMT"} })
14482025-06-03T00:51:41.229ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
14492025-06-03T00:51:41.229ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
14502025-06-03T00:51:41.229ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
14512025-06-03T00:51:41.230ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 722.03844ms iteration = 2 status = {}
14522025-06-03T00:51:41.232ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
14532025-06-03T00:51:41.233ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
14542025-06-03T00:51:41.235ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
14552025-06-03T00:51:41.235ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
14562025-06-03T00:51:41.238ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
14572025-06-03T00:51:41.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
14582025-06-03T00:51:41.239ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
14592025-06-03T00:51:41.240ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
14602025-06-03T00:51:41.242ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
14612025-06-03T00:51:41.242ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
14622025-06-03T00:51:41.243ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae switch_location = Switch0
14632025-06-03T00:51:41.243ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
14642025-06-03T00:51:41.252ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 743.814306ms iteration = 2 status = 6
14652025-06-03T00:51:41.280ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
14662025-06-03T00:51:41.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
14672025-06-03T00:51:41.324ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
14682025-06-03T00:51:41.324ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
14692025-06-03T00:51:41.363ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14702025-06-03T00:51:41.367ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = VpcRouter { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000001, lookup_type: ById(001de000-074c-4000-8000-000000000001) } result = Ok(())
14712025-06-03T00:51:41.367ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
14722025-06-03T00:51:41.367ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
14732025-06-03T00:51:41.368ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
14742025-06-03T00:51:41.369ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
14752025-06-03T00:51:41.411ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
14762025-06-03T00:51:41.411ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
14772025-06-03T00:51:41.413ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14782025-06-03T00:51:41.417ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = VpcRouter { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000001, lookup_type: ById(001de000-074c-4000-8000-000000000001) } result = Ok(())
14792025-06-03T00:51:41.417ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
14802025-06-03T00:51:41.417ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
14812025-06-03T00:51:41.422ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
14822025-06-03T00:51:41.422ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
14832025-06-03T00:51:41.424ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
14842025-06-03T00:51:41.425ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
14852025-06-03T00:51:41.427ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
14862025-06-03T00:51:41.427ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
14872025-06-03T00:51:41.429ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
14882025-06-03T00:51:41.429ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
14892025-06-03T00:51:41.432ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
14902025-06-03T00:51:41.432ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
14912025-06-03T00:51:41.434ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
14922025-06-03T00:51:41.434ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
14932025-06-03T00:51:41.435ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
14942025-06-03T00:51:41.436ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
14952025-06-03T00:51:41.437ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14962025-06-03T00:51:41.438ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "deab4dad-e7e4-4004-bc4a-88992f381989", "date": "Tue, 03 Jun 2025 00:51:41 GMT"} })
14972025-06-03T00:51:41.438ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae switch = Switch0
14982025-06-03T00:51:41.438ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
14992025-06-03T00:51:41.439ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "7c8c6f82-04c8-4ec3-8e12-0df2c8d958e1", "content-length": "682", "date": "Tue, 03 Jun 2025 00:51:41 GMT"} })
15002025-06-03T00:51:41.439ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
15012025-06-03T00:51:41.439ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
15022025-06-03T00:51:41.442ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }, key: 001de000-c470-4000-8000-000000000003, lookup_type: ById(001de000-c470-4000-8000-000000000003) } result = Ok(())
15032025-06-03T00:51:41.442ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
15042025-06-03T00:51:41.442ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
15052025-06-03T00:51:41.445ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
15062025-06-03T00:51:41.445ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
15072025-06-03T00:51:41.447ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
15082025-06-03T00:51:41.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
15092025-06-03T00:51:41.449ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
15102025-06-03T00:51:41.449ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
15112025-06-03T00:51:41.451ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
15122025-06-03T00:51:41.452ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
15132025-06-03T00:51:41.464ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
15142025-06-03T00:51:41.465ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
15152025-06-03T00:51:41.470ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
15162025-06-03T00:51:41.470ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
15172025-06-03T00:51:41.472ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
15182025-06-03T00:51:41.473ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
15192025-06-03T00:51:41.487ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
15202025-06-03T00:51:41.488ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
15212025-06-03T00:51:41.489ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
15222025-06-03T00:51:41.489ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
15232025-06-03T00:51:41.491ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15242025-06-03T00:51:41.496ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ById(001de000-c470-4000-8000-000000000001) } result = Ok(())
15252025-06-03T00:51:41.496ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
15262025-06-03T00:51:41.496ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
15272025-06-03T00:51:41.498ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
15282025-06-03T00:51:41.498ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
15292025-06-03T00:51:41.500ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
15302025-06-03T00:51:41.500ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
15312025-06-03T00:51:41.501ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 993.152409ms iteration = 2 status = {}
15322025-06-03T00:51:41.502ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
15332025-06-03T00:51:41.502ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
15342025-06-03T00:51:41.504ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
15352025-06-03T00:51:41.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
15362025-06-03T00:51:41.505ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
15372025-06-03T00:51:41.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
15382025-06-03T00:51:41.507ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
15392025-06-03T00:51:41.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
15402025-06-03T00:51:41.509ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15412025-06-03T00:51:41.513ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ById(001de000-c470-4000-8000-000000000002) } result = Ok(())
15422025-06-03T00:51:41.513ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
15432025-06-03T00:51:41.513ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
15442025-06-03T00:51:41.515ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
15452025-06-03T00:51:41.515ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
15462025-06-03T00:51:41.516ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
15472025-06-03T00:51:41.516ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
15482025-06-03T00:51:41.518ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
15492025-06-03T00:51:41.519ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
15502025-06-03T00:51:41.521ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
15512025-06-03T00:51:41.521ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
15522025-06-03T00:51:41.522ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
15532025-06-03T00:51:41.523ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
15542025-06-03T00:51:41.524ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
15552025-06-03T00:51:41.524ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
15562025-06-03T00:51:41.525ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15572025-06-03T00:51:41.529ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
15582025-06-03T00:51:41.529ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
15592025-06-03T00:51:41.530ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
15602025-06-03T00:51:41.532ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
15612025-06-03T00:51:41.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
15622025-06-03T00:51:41.534ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
15632025-06-03T00:51:41.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
15642025-06-03T00:51:41.536ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
15652025-06-03T00:51:41.536ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
15662025-06-03T00:51:41.539ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
15672025-06-03T00:51:41.539ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
15682025-06-03T00:51:41.540ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
15692025-06-03T00:51:41.540ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
15702025-06-03T00:51:41.542ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
15712025-06-03T00:51:41.542ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
15722025-06-03T00:51:41.543ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15732025-06-03T00:51:41.547ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }, key: 001de000-c470-4000-8000-000000000003, lookup_type: ByName("boundary-ntp") } result = Ok(())
15742025-06-03T00:51:41.547ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
15752025-06-03T00:51:41.547ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
15762025-06-03T00:51:41.549ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
15772025-06-03T00:51:41.549ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
15782025-06-03T00:51:41.550ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
15792025-06-03T00:51:41.551ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
15802025-06-03T00:51:41.552ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
15812025-06-03T00:51:41.552ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
15822025-06-03T00:51:41.554ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
15832025-06-03T00:51:41.554ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
15842025-06-03T00:51:41.556ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
15852025-06-03T00:51:41.556ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
15862025-06-03T00:51:41.557ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
15872025-06-03T00:51:41.558ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
15882025-06-03T00:51:41.559ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15892025-06-03T00:51:41.563ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
15902025-06-03T00:51:41.563ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
15912025-06-03T00:51:41.563ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
15922025-06-03T00:51:41.564ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
15932025-06-03T00:51:41.564ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
15942025-06-03T00:51:41.566ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
15952025-06-03T00:51:41.566ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
15962025-06-03T00:51:41.567ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
15972025-06-03T00:51:41.567ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
15982025-06-03T00:51:41.570ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
15992025-06-03T00:51:41.570ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
16002025-06-03T00:51:41.571ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
16012025-06-03T00:51:41.571ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
16022025-06-03T00:51:41.573ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
16032025-06-03T00:51:41.573ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
16042025-06-03T00:51:41.574ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16052025-06-03T00:51:41.578ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = InternetGateway { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000002, lookup_type: ByName("default") } result = Ok(())
16062025-06-03T00:51:41.581ZDEBGnexus (BackgroundTasks): client request SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
16072025-06-03T00:51:41.581ZDEBGnexus (BackgroundTasks): client response SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "443b4cab-4dce-4d0b-82e3-f2d7ccc5c01a", "date": "Tue, 03 Jun 2025 00:51:41 GMT"} })
16082025-06-03T00:51:41.583ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 1.074492244s iteration = 2 status = {}
16092025-06-03T00:51:41.833ZDEBGnexus (BackgroundTasks): client response background_task = crdb_node_id_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "7facc25f-4766-4839-8582-571928248bb3", "content-length": "64", "date": "Tue, 03 Jun 2025 00:51:41 GMT"} })
16102025-06-03T00:51:41.833ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
16112025-06-03T00:51:41.833ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
16122025-06-03T00:51:41.835ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16132025-06-03T00:51:41.837ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
16142025-06-03T00:51:41.837ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
16152025-06-03T00:51:41.837ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
16162025-06-03T00:51:41.892ZDEBGnexus (BackgroundTasks): client response background_task = crdb_node_id_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "779abe9a-bed8-4eaa-b3dd-bfc05ce8e98a", "content-length": "64", "date": "Tue, 03 Jun 2025 00:51:41 GMT"} })
16172025-06-03T00:51:41.892ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
16182025-06-03T00:51:41.892ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
16192025-06-03T00:51:41.894ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16202025-06-03T00:51:41.896ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
16212025-06-03T00:51:41.896ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
16222025-06-03T00:51:41.896ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
16232025-06-03T00:51:41.926ZDEBGnexus (BackgroundTasks): client response background_task = crdb_node_id_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "eed0e563-cd57-4897-9b4b-92f2831d3e36", "content-length": "64", "date": "Tue, 03 Jun 2025 00:51:41 GMT"} })
16242025-06-03T00:51:41.926ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
16252025-06-03T00:51:41.926ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
16262025-06-03T00:51:41.928ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16272025-06-03T00:51:41.929ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
16282025-06-03T00:51:41.929ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
16292025-06-03T00:51:41.929ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
16302025-06-03T00:51:41.970ZDEBGnexus (BackgroundTasks): client response background_task = crdb_node_id_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "30b0ce4a-6fdb-4b01-bd9a-dd8a6775baf2", "content-length": "64", "date": "Tue, 03 Jun 2025 00:51:41 GMT"} })
16312025-06-03T00:51:41.970ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
16322025-06-03T00:51:41.971ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
16332025-06-03T00:51:41.972ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16342025-06-03T00:51:41.974ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
16352025-06-03T00:51:41.974ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
16362025-06-03T00:51:41.974ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
16372025-06-03T00:51:42.006ZDEBGnexus (BackgroundTasks): client response background_task = crdb_node_id_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "d7a79f13-108c-4116-a489-abab56156c9d", "content-length": "64", "date": "Tue, 03 Jun 2025 00:51:42 GMT"} })
16382025-06-03T00:51:42.006ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
16392025-06-03T00:51:42.006ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
16402025-06-03T00:51:42.007ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16412025-06-03T00:51:42.008ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
16422025-06-03T00:51:42.008ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
16432025-06-03T00:51:42.008ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
16442025-06-03T00:51:42.015ZDEBGnexus (BackgroundTasks): activation complete background_task = crdb_node_id_collector elapsed = 1.401031383s iteration = 2 status = {"nsuccess":5}
16452025-06-03T00:51:50.508ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 5 reason = Timeout
16462025-06-03T00:51:50.508ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 5 reason = Timeout
16472025-06-03T00:51:50.508ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
16482025-06-03T00:51:50.508ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
16492025-06-03T00:51:50.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
16502025-06-03T00:51:50.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
16512025-06-03T00:51:50.510ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16522025-06-03T00:51:50.513ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
16532025-06-03T00:51:50.513ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
16542025-06-03T00:51:50.513ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
16552025-06-03T00:51:50.516ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
16562025-06-03T00:51:50.517ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
16572025-06-03T00:51:50.518ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16582025-06-03T00:51:50.521ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 4ec93699-353f-4b9e-ae06-25355fdb069e, lookup_type: ById(4ec93699-353f-4b9e-ae06-25355fdb069e) } result = Ok(())
16592025-06-03T00:51:50.521ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
16602025-06-03T00:51:50.521ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
16612025-06-03T00:51:50.532ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 4ec93699-353f-4b9e-ae06-25355fdb069e original_time_created = 2025-06-03 00:51:07.059326 UTC target_id = 4ec93699-353f-4b9e-ae06-25355fdb069e time_created = 2025-06-03 00:51:07.059326 UTC
16622025-06-03T00:51:50.534ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 26.501853ms iteration = 5 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"4ec93699-353f-4b9e-ae06-25355fdb069e","time_created":"2025-06-03T00:51:07.059326Z"}
16632025-06-03T00:51:50.546ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
16642025-06-03T00:51:50.546ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
16652025-06-03T00:51:50.579ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 71.001905ms iteration = 5 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
16662025-06-03T00:52:00.508ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 6 reason = Timeout
16672025-06-03T00:52:00.508ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
16682025-06-03T00:52:00.508ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 6 reason = Timeout
16692025-06-03T00:52:00.508ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
16702025-06-03T00:52:00.509ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
16712025-06-03T00:52:00.509ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
16722025-06-03T00:52:00.511ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
16732025-06-03T00:52:00.511ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
16742025-06-03T00:52:00.511ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16752025-06-03T00:52:00.514ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
16762025-06-03T00:52:00.514ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
16772025-06-03T00:52:00.514ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
16782025-06-03T00:52:00.516ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
16792025-06-03T00:52:00.516ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
16802025-06-03T00:52:00.518ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16812025-06-03T00:52:00.521ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 4ec93699-353f-4b9e-ae06-25355fdb069e, lookup_type: ById(4ec93699-353f-4b9e-ae06-25355fdb069e) } result = Ok(())
16822025-06-03T00:52:00.521ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
16832025-06-03T00:52:00.521ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
16842025-06-03T00:52:00.535ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 4ec93699-353f-4b9e-ae06-25355fdb069e original_time_created = 2025-06-03 00:51:07.059326 UTC target_id = 4ec93699-353f-4b9e-ae06-25355fdb069e time_created = 2025-06-03 00:51:07.059326 UTC
16852025-06-03T00:52:00.535ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 26.317397ms iteration = 6 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"4ec93699-353f-4b9e-ae06-25355fdb069e","time_created":"2025-06-03T00:51:07.059326Z"}
16862025-06-03T00:52:00.544ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 35.201293ms iteration = 6 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
16872025-06-03T00:52:10.507ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 2 reason = Timeout
16882025-06-03T00:52:10.507ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
16892025-06-03T00:52:10.507ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
16902025-06-03T00:52:10.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
16912025-06-03T00:52:10.508ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 7 reason = Timeout
16922025-06-03T00:52:10.508ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
16932025-06-03T00:52:10.508ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 3 reason = Timeout
16942025-06-03T00:52:10.508ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
16952025-06-03T00:52:10.508ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 3 reason = Timeout
16962025-06-03T00:52:10.508ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
16972025-06-03T00:52:10.508ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 3 reason = Timeout
16982025-06-03T00:52:10.508ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
16992025-06-03T00:52:10.508ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
17002025-06-03T00:52:10.508ZDEBGnexus (BackgroundTasks): activating background_task = webhook_deliverator iteration = 2 reason = Timeout
17012025-06-03T00:52:10.508ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 3 reason = Timeout
17022025-06-03T00:52:10.508ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
17032025-06-03T00:52:10.508ZDEBGnexus (BackgroundTasks): roles background_task = webhook_deliverator roles = RoleSet { roles: {} }
17042025-06-03T00:52:10.508ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 2 reason = Timeout
17052025-06-03T00:52:10.508ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 2 reason = Timeout
17062025-06-03T00:52:10.508ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-06-03 00:42:10.508339695 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
17072025-06-03T00:52:10.508ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 3 reason = Timeout
17082025-06-03T00:52:10.508ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
17092025-06-03T00:52:10.508ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
17102025-06-03T00:52:10.508ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 3 reason = Timeout
17112025-06-03T00:52:10.508ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
17122025-06-03T00:52:10.508ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 3 reason = Timeout
17132025-06-03T00:52:10.508ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
17142025-06-03T00:52:10.508ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 3 reason = Timeout
17152025-06-03T00:52:10.508ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 3 reason = Timeout
17162025-06-03T00:52:10.508ZWARNnexus (BackgroundTasks): DNS propagation: skipped background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:69 reason = no config
17172025-06-03T00:52:10.508ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 8.09\u{b5}s iteration = 3 status = {"error":"no config"}
17182025-06-03T00:52:10.508ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 4 reason = Timeout
17192025-06-03T00:52:10.508ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
17202025-06-03T00:52:10.508ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 3 reason = Timeout
17212025-06-03T00:52:10.508ZWARNnexus (BackgroundTasks): DNS propagation: skipped background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:69 reason = no config
17222025-06-03T00:52:10.508ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 6.34\u{b5}s iteration = 3 status = {"error":"no config"}
17232025-06-03T00:52:10.508ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
17242025-06-03T00:52:10.508ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 3 reason = Timeout
17252025-06-03T00:52:10.508ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 2 reason = Timeout
17262025-06-03T00:52:10.508ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:85 reason = blueprint disabled target_id = 4ec93699-353f-4b9e-ae06-25355fdb069e
17272025-06-03T00:52:10.508ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
17282025-06-03T00:52:10.508ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 13.32\u{b5}s iteration = 3 status = {"enabled":false,"target_id":"4ec93699-353f-4b9e-ae06-25355fdb069e"}
17292025-06-03T00:52:10.508ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
17302025-06-03T00:52:10.508ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 2 reason = Timeout
17312025-06-03T00:52:10.508ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 3 reason = Timeout
17322025-06-03T00:52:10.508ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
17332025-06-03T00:52:10.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
17342025-06-03T00:52:10.508ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 3 reason = Timeout
17352025-06-03T00:52:10.508ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
17362025-06-03T00:52:10.508ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 3 reason = Timeout
17372025-06-03T00:52:10.508ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
17382025-06-03T00:52:10.508ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 3 reason = Timeout
17392025-06-03T00:52:10.508ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
17402025-06-03T00:52:10.508ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
17412025-06-03T00:52:10.508ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 2 reason = Timeout
17422025-06-03T00:52:10.508ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
17432025-06-03T00:52:10.508ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 3 reason = Timeout
17442025-06-03T00:52:10.508ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 3 reason = Timeout
17452025-06-03T00:52:10.508ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
17462025-06-03T00:52:10.508ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 2 reason = Timeout
17472025-06-03T00:52:10.508ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 3 reason = Timeout
17482025-06-03T00:52:10.508ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
17492025-06-03T00:52:10.508ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
17502025-06-03T00:52:10.508ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
17512025-06-03T00:52:10.508ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 2 reason = Timeout
17522025-06-03T00:52:10.508ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
17532025-06-03T00:52:10.508ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 3 reason = Timeout
17542025-06-03T00:52:10.508ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
17552025-06-03T00:52:10.508ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 2 reason = Timeout
17562025-06-03T00:52:10.508ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
17572025-06-03T00:52:10.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
17582025-06-03T00:52:10.508ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 2 reason = Timeout
17592025-06-03T00:52:10.508ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:53
17602025-06-03T00:52:10.508ZDEBGnexus (BackgroundTasks): activating background_task = alert_dispatcher iteration = 2 reason = Timeout
17612025-06-03T00:52:10.508ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
17622025-06-03T00:52:10.508ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
17632025-06-03T00:52:10.508ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 7 reason = Timeout
17642025-06-03T00:52:10.508ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
17652025-06-03T00:52:10.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
17662025-06-03T00:52:10.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
17672025-06-03T00:52:10.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
17682025-06-03T00:52:10.509ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
17692025-06-03T00:52:10.509ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
17702025-06-03T00:52:10.509ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
17712025-06-03T00:52:10.509ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = webhook_deliverator resource = Database result = Ok(())
17722025-06-03T00:52:10.509ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
17732025-06-03T00:52:10.509ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
17742025-06-03T00:52:10.509ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
17752025-06-03T00:52:10.509ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
17762025-06-03T00:52:10.509ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
17772025-06-03T00:52:10.509ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
17782025-06-03T00:52:10.509ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
17792025-06-03T00:52:10.509ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
17802025-06-03T00:52:10.509ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
17812025-06-03T00:52:10.509ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
17822025-06-03T00:52:10.509ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
17832025-06-03T00:52:10.509ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
17842025-06-03T00:52:10.509ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
17852025-06-03T00:52:10.509ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
17862025-06-03T00:52:10.509ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
17872025-06-03T00:52:10.509ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
17882025-06-03T00:52:10.509ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
17892025-06-03T00:52:10.509ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("241573bc-467c-4802-9f4b-f7c82da4e12e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("64fc0a69-d4b8-4f6c-9a07-77a4cfa94962.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("ceb74144-4b6b-4b3f-bc80-eac2d75f3bc9.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("ceb74144-4b6b-4b3f-bc80-eac2d75f3bc9.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:2::1))) }], valid_until: Instant { tv_sec: 2457, tv_nsec: 731048710 } })
17902025-06-03T00:52:10.509ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("9feb16c9-ff60-43e8-917b-965124f46e0d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("aa124775-31de-4648-981a-4af500268b66.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("aa124775-31de-4648-981a-4af500268b66.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::9))) }], valid_until: Instant { tv_sec: 2457, tv_nsec: 731062090 } })
17912025-06-03T00:52:10.509ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
17922025-06-03T00:52:10.509ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = alert_dispatcher resource = Database result = Ok(())
17932025-06-03T00:52:10.509ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
17942025-06-03T00:52:10.509ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.461606ms iteration = 2 status = {"addresses":["[fd00:1122:3344:101::8]:5353","[fd00:1122:3344:101::9]:5353"]}
17952025-06-03T00:52:10.510ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } dns_group = internal
17962025-06-03T00:52:10.510ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.721633ms iteration = 2 status = {"addresses":["[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:3::1]:5353","[fd00:1122:3344:2::1]:5353"]}
17972025-06-03T00:52:10.510ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
17982025-06-03T00:52:10.510ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
17992025-06-03T00:52:10.513ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18002025-06-03T00:52:10.514ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18012025-06-03T00:52:10.515ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 6.658867ms iteration = 3 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
18022025-06-03T00:52:10.516ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
18032025-06-03T00:52:10.516ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
18042025-06-03T00:52:10.516ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
18052025-06-03T00:52:10.516ZINFOnexus (BackgroundTasks): all webhook delivery tasks completed background_task = webhook_deliverator file = nexus/src/app/background/tasks/webhook_deliverator.rs:202 num_receivers = 0
18062025-06-03T00:52:10.516ZDEBGnexus (BackgroundTasks): activation complete background_task = webhook_deliverator elapsed = 8.598758ms iteration = 2 status = {"by_rx":{},"error":null}
18072025-06-03T00:52:10.517ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-06-03 00:42:10.508339695 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
18082025-06-03T00:52:10.517ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 9.000779ms iteration = 2 status = {"expiration":"2025-06-03T00:42:10.508339695Z","pruned":[]}
18092025-06-03T00:52:10.518ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
18102025-06-03T00:52:10.521ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-afe89ed7-19b7-479b-81eb-05c988ebc66d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-afe89ed7-19b7-479b-81eb-05c988ebc66d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 2457, tv_nsec: 743477546 } })
18112025-06-03T00:52:10.522ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1179
18122025-06-03T00:52:10.525ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
18132025-06-03T00:52:10.525ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
18142025-06-03T00:52:10.532ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 23.762707ms iteration = 2 status = {"errors":[],"found_port_ok":[]}
18152025-06-03T00:52:10.534ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18162025-06-03T00:52:10.537ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
18172025-06-03T00:52:10.537ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
18182025-06-03T00:52:10.537ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
18192025-06-03T00:52:10.540ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-afe89ed7-19b7-479b-81eb-05c988ebc66d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-afe89ed7-19b7-479b-81eb-05c988ebc66d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 2457, tv_nsec: 761982711 } })
18202025-06-03T00:52:10.540ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1179
18212025-06-03T00:52:10.546ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18222025-06-03T00:52:10.550ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
18232025-06-03T00:52:10.550ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
18242025-06-03T00:52:10.550ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
18252025-06-03T00:52:10.570ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 62.516471ms iteration = 3 status = {"errors":[],"garbage_collect_requested":[]}
18262025-06-03T00:52:10.572ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 63.959486ms iteration = 3 status = {"errors":[],"finish_invoked_ok":[]}
18272025-06-03T00:52:10.583ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18282025-06-03T00:52:10.587ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
18292025-06-03T00:52:10.587ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
18302025-06-03T00:52:10.587ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
18312025-06-03T00:52:10.594ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 86.082553ms iteration = 3 status = {"errors":[],"requests_created_ok":[]}
18322025-06-03T00:52:10.597ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
18332025-06-03T00:52:10.597ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
18342025-06-03T00:52:10.597ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
18352025-06-03T00:52:10.597ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
18362025-06-03T00:52:10.597ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1187 zone_address = fd00:1122:3344:101::2
18372025-06-03T00:52:10.597ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
18382025-06-03T00:52:10.597ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
18392025-06-03T00:52:10.598ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "232fb6ff-924a-413c-ac12-f1498d0c3a2b", "content-length": "26", "date": "Tue, 03 Jun 2025 00:52:10 GMT"} })
18402025-06-03T00:52:10.598ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1190 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
18412025-06-03T00:52:10.598ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18422025-06-03T00:52:10.599ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1224 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
18432025-06-03T00:52:10.599ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18442025-06-03T00:52:10.601ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
18452025-06-03T00:52:10.601ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
18462025-06-03T00:52:10.602ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
18472025-06-03T00:52:10.602ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
18482025-06-03T00:52:10.602ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
18492025-06-03T00:52:10.602ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
18502025-06-03T00:52:10.602ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
18512025-06-03T00:52:10.602ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
18522025-06-03T00:52:10.602ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
18532025-06-03T00:52:10.602ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
18542025-06-03T00:52:10.603ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
18552025-06-03T00:52:10.603ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
18562025-06-03T00:52:10.603ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
18572025-06-03T00:52:10.603ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
18582025-06-03T00:52:10.604ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
18592025-06-03T00:52:10.604ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
18602025-06-03T00:52:10.604ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18612025-06-03T00:52:10.605ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18622025-06-03T00:52:10.605ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18632025-06-03T00:52:10.606ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18642025-06-03T00:52:10.606ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18652025-06-03T00:52:10.607ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 98.999024ms iteration = 7 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
18662025-06-03T00:52:10.607ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18672025-06-03T00:52:10.607ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
18682025-06-03T00:52:10.607ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
18692025-06-03T00:52:10.607ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
18702025-06-03T00:52:10.608ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
18712025-06-03T00:52:10.608ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:167 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
18722025-06-03T00:52:10.608ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 99.882881ms iteration = 2 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
18732025-06-03T00:52:10.609ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
18742025-06-03T00:52:10.609ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
18752025-06-03T00:52:10.609ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
18762025-06-03T00:52:10.609ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
18772025-06-03T00:52:10.609ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = alert_dispatcher resource = Database result = Ok(())
18782025-06-03T00:52:10.610ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
18792025-06-03T00:52:10.610ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
18802025-06-03T00:52:10.610ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
18812025-06-03T00:52:10.611ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
18822025-06-03T00:52:10.611ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
18832025-06-03T00:52:10.611ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
18842025-06-03T00:52:10.611ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
18852025-06-03T00:52:10.612ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
18862025-06-03T00:52:10.612ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
18872025-06-03T00:52:10.612ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
18882025-06-03T00:52:10.612ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
18892025-06-03T00:52:10.612ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
18902025-06-03T00:52:10.616ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
18912025-06-03T00:52:10.616ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
18922025-06-03T00:52:10.616ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18932025-06-03T00:52:10.620ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 4ec93699-353f-4b9e-ae06-25355fdb069e, lookup_type: ById(4ec93699-353f-4b9e-ae06-25355fdb069e) } result = Ok(())
18942025-06-03T00:52:10.620ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
18952025-06-03T00:52:10.620ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
18962025-06-03T00:52:10.624ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-afe89ed7-19b7-479b-81eb-05c988ebc66d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-afe89ed7-19b7-479b-81eb-05c988ebc66d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 2457, tv_nsec: 845867515 } })
18972025-06-03T00:52:10.624ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1179 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae
18982025-06-03T00:52:10.625ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
18992025-06-03T00:52:10.625ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
19002025-06-03T00:52:10.632ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1187 zone_address = fd00:1122:3344:101::2
19012025-06-03T00:52:10.632ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
19022025-06-03T00:52:10.632ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19032025-06-03T00:52:10.633ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "76fd704f-f5f9-4be2-aa5f-4b67dce3b636", "content-length": "26", "date": "Tue, 03 Jun 2025 00:52:10 GMT"} })
19042025-06-03T00:52:10.633ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1190 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
19052025-06-03T00:52:10.634ZDEBGnexus (BackgroundTasks): activation complete background_task = alert_dispatcher elapsed = 125.371742ms iteration = 2 status = {"dispatched":[],"errors":[],"glob_version":"144.0.0","globs_reprocessed":{},"no_receivers":[]}
19062025-06-03T00:52:10.636ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1224 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
19072025-06-03T00:52:10.636ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
19082025-06-03T00:52:10.641ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 4ec93699-353f-4b9e-ae06-25355fdb069e original_time_created = 2025-06-03 00:51:07.059326 UTC target_id = 4ec93699-353f-4b9e-ae06-25355fdb069e time_created = 2025-06-03 00:51:07.059326 UTC
19092025-06-03T00:52:10.641ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 132.969439ms iteration = 7 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"4ec93699-353f-4b9e-ae06-25355fdb069e","time_created":"2025-06-03T00:51:07.059326Z"}
19102025-06-03T00:52:10.641ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:547 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
19112025-06-03T00:52:10.641ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 133.697831ms iteration = 3 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
19122025-06-03T00:52:10.666ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
19132025-06-03T00:52:10.667ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
19142025-06-03T00:52:10.670ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
19152025-06-03T00:52:10.670ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
19162025-06-03T00:52:10.670ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
19172025-06-03T00:52:10.670ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
19182025-06-03T00:52:10.672ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19192025-06-03T00:52:10.673ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
19202025-06-03T00:52:10.673ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
19212025-06-03T00:52:10.673ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
19222025-06-03T00:52:10.673ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
19232025-06-03T00:52:10.674ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
19242025-06-03T00:52:10.674ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:137 found = 0 not_ready_to_be_deleted = 0
19252025-06-03T00:52:10.674ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 165.494859ms iteration = 2 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
19262025-06-03T00:52:10.675ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
19272025-06-03T00:52:10.675ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
19282025-06-03T00:52:10.675ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
19292025-06-03T00:52:10.675ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 166.771359ms iteration = 3 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
19302025-06-03T00:52:10.679ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
19312025-06-03T00:52:10.680ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
19322025-06-03T00:52:10.680ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 171.799455ms iteration = 3 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
19332025-06-03T00:52:10.681ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19342025-06-03T00:52:10.683ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
19352025-06-03T00:52:10.683ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
19362025-06-03T00:52:10.683ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
19372025-06-03T00:52:10.698ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
19382025-06-03T00:52:10.698ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
19392025-06-03T00:52:10.698ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
19402025-06-03T00:52:10.698ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
19412025-06-03T00:52:10.698ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
19422025-06-03T00:52:10.700ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19432025-06-03T00:52:10.703ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
19442025-06-03T00:52:10.703ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
19452025-06-03T00:52:10.703ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
19462025-06-03T00:52:10.703ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
19472025-06-03T00:52:10.703ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
19482025-06-03T00:52:10.709ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
19492025-06-03T00:52:10.709ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
19502025-06-03T00:52:10.709ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19512025-06-03T00:52:10.714ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: afe89ed7-19b7-479b-81eb-05c988ebc66d, lookup_type: ById(afe89ed7-19b7-479b-81eb-05c988ebc66d) } result = Ok(())
19522025-06-03T00:52:10.714ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
19532025-06-03T00:52:10.714ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
19542025-06-03T00:52:10.715ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
19552025-06-03T00:52:10.715ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
19562025-06-03T00:52:10.715ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
19572025-06-03T00:52:10.716ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
19582025-06-03T00:52:10.717ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 208.682049ms iteration = 3 status = 0
19592025-06-03T00:52:10.717ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19602025-06-03T00:52:10.720ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
19612025-06-03T00:52:10.720ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
19622025-06-03T00:52:10.720ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
19632025-06-03T00:52:10.720ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
19642025-06-03T00:52:10.720ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
19652025-06-03T00:52:10.720ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 212.258191ms iteration = 3 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
19662025-06-03T00:52:10.727ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:372
19672025-06-03T00:52:10.727ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 219.461004ms iteration = 3 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
19682025-06-03T00:52:10.728ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1187 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae zone_address = fd00:1122:3344:101::2
19692025-06-03T00:52:10.728ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
19702025-06-03T00:52:10.729ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "d39a0310-6ffe-4c6a-80f2-355c7f2b198c", "content-length": "26", "date": "Tue, 03 Jun 2025 00:52:10 GMT"} })
19712025-06-03T00:52:10.729ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1190 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
19722025-06-03T00:52:10.731ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1224 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae
19732025-06-03T00:52:10.743ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
19742025-06-03T00:52:10.744ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
19752025-06-03T00:52:10.769ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 260.726986ms iteration = 3 status = {}
19762025-06-03T00:52:10.780ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
19772025-06-03T00:52:10.780ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 9e79f9b1-a4de-4f9d-a901-16cc4f79e15d dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
19782025-06-03T00:52:10.780ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
19792025-06-03T00:52:10.780ZINFOnexus (BackgroundTasks): found Silo/DNS/TLS config (initial) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-06-03T00:51:07.585932Z, time_modified: 2025-06-03T00:51:07.585932Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: ea0ef520-6bf3-4fbd-9ff0-f3698b48b749, db_silo: Silo { identity: SiloIdentity { id: ea0ef520-6bf3-4fbd-9ff0-f3698b48b749, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-06-03T00:51:38.523369Z, time_modified: 2025-06-03T00:51:38.523369Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "bcda8a9f16e63c3cec49f9759478b9f335a661639158b0a9026f69b846d4e127" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: ea0ef520-6bf3-4fbd-9ff0-f3698b48b749, db_silo: Silo { identity: SiloIdentity { id: ea0ef520-6bf3-4fbd-9ff0-f3698b48b749, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-06-03T00:51:38.523369Z, time_modified: 2025-06-03T00:51:38.523369Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "bcda8a9f16e63c3cec49f9759478b9f335a661639158b0a9026f69b846d4e127" }] }) } file = nexus/src/app/background/tasks/external_endpoints.rs:77
19802025-06-03T00:52:10.780ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 272.244003ms iteration = 2 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"ea0ef520-6bf3-4fbd-9ff0-f3698b48b749","tls_certs":["bcda8a9f16e63c3cec49f9759478b9f335a661639158b0a9026f69b846d4e127"]}},"default_endpoint":{"silo_id":"ea0ef520-6bf3-4fbd-9ff0-f3698b48b749","tls_certs":["bcda8a9f16e63c3cec49f9759478b9f335a661639158b0a9026f69b846d4e127"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
19812025-06-03T00:52:10.781ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
19822025-06-03T00:52:10.793ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 284.976399ms iteration = 4 status = {"physical_disks_added":0}
19832025-06-03T00:52:10.833ZINFOnexus (BackgroundTasks): VPC route manager sled afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
19842025-06-03T00:52:10.833ZDEBGnexus (BackgroundTasks): client request SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
19852025-06-03T00:52:10.834ZDEBGnexus (BackgroundTasks): client response SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "8834c70c-ccb3-4fc3-8c6b-5631f11620bd", "content-length": "372", "date": "Tue, 03 Jun 2025 00:52:10 GMT"} })
19862025-06-03T00:52:10.834ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
19872025-06-03T00:52:10.834ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
19882025-06-03T00:52:10.835ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19892025-06-03T00:52:10.840ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = / versions = all
19902025-06-03T00:52:10.840ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /access versions = all
19912025-06-03T00:52:10.840ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /assets/{path} versions = all
19922025-06-03T00:52:10.840ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /device/auth versions = all
19932025-06-03T00:52:10.840ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /device/confirm versions = all
19942025-06-03T00:52:10.840ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /device/success versions = all
19952025-06-03T00:52:10.840ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /device/token versions = all
19962025-06-03T00:52:10.840ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /device/verify versions = all
19972025-06-03T00:52:10.840ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /experimental/v1/probes versions = all
19982025-06-03T00:52:10.840ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
19992025-06-03T00:52:10.840ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /experimental/v1/probes versions = all
20002025-06-03T00:52:10.840ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /experimental/v1/probes/{probe} versions = all
20012025-06-03T00:52:10.840ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /experimental/v1/probes/{probe} versions = all
20022025-06-03T00:52:10.840ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
20032025-06-03T00:52:10.840ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /experimental/v1/system/support-bundles versions = all
20042025-06-03T00:52:10.840ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /experimental/v1/system/support-bundles versions = all
20052025-06-03T00:52:10.840ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /experimental/v1/system/support-bundles/{bundle_id} versions = all
20062025-06-03T00:52:10.840ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /experimental/v1/system/support-bundles/{bundle_id} versions = all
20072025-06-03T00:52:10.840ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /experimental/v1/system/support-bundles/{bundle_id}/download versions = all
20082025-06-03T00:52:10.840ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = HEAD path = /experimental/v1/system/support-bundles/{bundle_id}/download versions = all
20092025-06-03T00:52:10.840ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /experimental/v1/system/support-bundles/{bundle_id}/download/{file} versions = all
20102025-06-03T00:52:10.840ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = HEAD path = /experimental/v1/system/support-bundles/{bundle_id}/download/{file} versions = all
20112025-06-03T00:52:10.840ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /experimental/v1/system/support-bundles/{bundle_id}/index versions = all
20122025-06-03T00:52:10.840ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /images versions = all
20132025-06-03T00:52:10.840ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /login versions = all
20142025-06-03T00:52:10.840ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /login/{silo_name}/local versions = all
20152025-06-03T00:52:10.840ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /login/{silo_name}/saml/{provider_name} versions = all
20162025-06-03T00:52:10.840ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /login/{silo_name}/saml/{provider_name} versions = all
20172025-06-03T00:52:10.840ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /login/{silo_name}/saml/{provider_name}/redirect versions = all
20182025-06-03T00:52:10.840ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /lookup/{path} versions = all
20192025-06-03T00:52:10.840ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /projects/{path} versions = all
20202025-06-03T00:52:10.840ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /projects-new versions = all
20212025-06-03T00:52:10.840ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /settings/{path} versions = all
20222025-06-03T00:52:10.840ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /system/{path} versions = all
20232025-06-03T00:52:10.840ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /utilization versions = all
20242025-06-03T00:52:10.840ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/affinity-groups versions = all
20252025-06-03T00:52:10.840ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/affinity-groups versions = all
20262025-06-03T00:52:10.840ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/affinity-groups/{affinity_group} versions = all
20272025-06-03T00:52:10.840ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/affinity-groups/{affinity_group} versions = all
20282025-06-03T00:52:10.840ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = PUT path = /v1/affinity-groups/{affinity_group} versions = all
20292025-06-03T00:52:10.840ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/affinity-groups/{affinity_group}/members versions = all
20302025-06-03T00:52:10.840ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/affinity-groups/{affinity_group}/members/instance/{instance} versions = all
20312025-06-03T00:52:10.840ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/affinity-groups/{affinity_group}/members/instance/{instance} versions = all
20322025-06-03T00:52:10.840ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/affinity-groups/{affinity_group}/members/instance/{instance} versions = all
20332025-06-03T00:52:10.840ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/alert-classes versions = all
20342025-06-03T00:52:10.840ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/alert-receivers versions = all
20352025-06-03T00:52:10.840ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/alert-receivers/{receiver} versions = all
20362025-06-03T00:52:10.840ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/alert-receivers/{receiver} versions = all
20372025-06-03T00:52:10.840ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/alert-receivers/{receiver}/deliveries versions = all
20382025-06-03T00:52:10.840ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/alert-receivers/{receiver}/probe versions = all
20392025-06-03T00:52:10.840ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/alert-receivers/{receiver}/subscriptions versions = all
20402025-06-03T00:52:10.840ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/alert-receivers/{receiver}/subscriptions/{subscription} versions = all
20412025-06-03T00:52:10.840ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/alerts/{alert_id}/resend versions = all
20422025-06-03T00:52:10.840ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/anti-affinity-groups versions = all
20432025-06-03T00:52:10.840ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/anti-affinity-groups versions = all
20442025-06-03T00:52:10.840ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/anti-affinity-groups/{anti_affinity_group} versions = all
20452025-06-03T00:52:10.840ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/anti-affinity-groups/{anti_affinity_group} versions = all
20462025-06-03T00:52:10.840ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = PUT path = /v1/anti-affinity-groups/{anti_affinity_group} versions = all
20472025-06-03T00:52:10.840ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/anti-affinity-groups/{anti_affinity_group}/members versions = all
20482025-06-03T00:52:10.840ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/anti-affinity-groups/{anti_affinity_group}/members/instance/{instance} versions = all
20492025-06-03T00:52:10.840ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/anti-affinity-groups/{anti_affinity_group}/members/instance/{instance} versions = all
20502025-06-03T00:52:10.840ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/anti-affinity-groups/{anti_affinity_group}/members/instance/{instance} versions = all
20512025-06-03T00:52:10.840ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/certificates versions = all
20522025-06-03T00:52:10.840ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/certificates versions = all
20532025-06-03T00:52:10.840ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/certificates/{certificate} versions = all
20542025-06-03T00:52:10.840ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/certificates/{certificate} versions = all
20552025-06-03T00:52:10.840ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/disks versions = all
20562025-06-03T00:52:10.840ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/disks versions = all
20572025-06-03T00:52:10.840ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/disks/{disk} versions = all
20582025-06-03T00:52:10.840ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/disks/{disk} versions = all
20592025-06-03T00:52:10.840ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/disks/{disk}/bulk-write versions = all
20602025-06-03T00:52:10.840ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/disks/{disk}/bulk-write-start versions = all
20612025-06-03T00:52:10.840ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/disks/{disk}/bulk-write-stop versions = all
20622025-06-03T00:52:10.840ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/disks/{disk}/finalize versions = all
20632025-06-03T00:52:10.840ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/disks/{disk}/metrics/{metric} versions = all
20642025-06-03T00:52:10.840ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/floating-ips versions = all
20652025-06-03T00:52:10.840ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/floating-ips versions = all
20662025-06-03T00:52:10.840ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/floating-ips/{floating_ip} versions = all
20672025-06-03T00:52:10.840ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/floating-ips/{floating_ip} versions = all
20682025-06-03T00:52:10.840ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = PUT path = /v1/floating-ips/{floating_ip} versions = all
20692025-06-03T00:52:10.841ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/floating-ips/{floating_ip}/attach versions = all
20702025-06-03T00:52:10.841ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/floating-ips/{floating_ip}/detach versions = all
20712025-06-03T00:52:10.841ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/groups versions = all
20722025-06-03T00:52:10.841ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/groups/{group_id} versions = all
20732025-06-03T00:52:10.841ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/images versions = all
20742025-06-03T00:52:10.841ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/images versions = all
20752025-06-03T00:52:10.841ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/images/{image} versions = all
20762025-06-03T00:52:10.841ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/images/{image} versions = all
20772025-06-03T00:52:10.841ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/images/{image}/demote versions = all
20782025-06-03T00:52:10.841ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/images/{image}/promote versions = all
20792025-06-03T00:52:10.841ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/instances versions = all
20802025-06-03T00:52:10.841ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/instances versions = all
20812025-06-03T00:52:10.841ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/instances/{instance} versions = all
20822025-06-03T00:52:10.841ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/instances/{instance} versions = all
20832025-06-03T00:52:10.841ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = PUT path = /v1/instances/{instance} versions = all
20842025-06-03T00:52:10.841ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/instances/{instance}/affinity-groups versions = all
20852025-06-03T00:52:10.841ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
20862025-06-03T00:52:10.841ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/instances/{instance}/anti-affinity-groups versions = all
20872025-06-03T00:52:10.841ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/instances/{instance}/disks versions = all
20882025-06-03T00:52:10.841ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/instances/{instance}/disks/attach versions = all
20892025-06-03T00:52:10.841ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/instances/{instance}/disks/detach versions = all
20902025-06-03T00:52:10.841ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/instances/{instance}/external-ips versions = all
20912025-06-03T00:52:10.841ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/instances/{instance}/external-ips/ephemeral versions = all
20922025-06-03T00:52:10.841ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/instances/{instance}/external-ips/ephemeral versions = all
20932025-06-03T00:52:10.841ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/instances/{instance}/reboot versions = all
20942025-06-03T00:52:10.841ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/instances/{instance}/serial-console versions = all
20952025-06-03T00:52:10.841ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/instances/{instance}/serial-console/stream versions = all
20962025-06-03T00:52:10.841ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/instances/{instance}/ssh-public-keys versions = all
20972025-06-03T00:52:10.841ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/instances/{instance}/start versions = all
20982025-06-03T00:52:10.841ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/instances/{instance}/stop versions = all
20992025-06-03T00:52:10.841ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/internet-gateway-ip-addresses versions = all
21002025-06-03T00:52:10.841ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/internet-gateway-ip-addresses versions = all
21012025-06-03T00:52:10.841ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/internet-gateway-ip-addresses/{address} versions = all
21022025-06-03T00:52:10.841ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/internet-gateway-ip-pools versions = all
21032025-06-03T00:52:10.841ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/internet-gateway-ip-pools versions = all
21042025-06-03T00:52:10.841ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/internet-gateway-ip-pools/{pool} versions = all
21052025-06-03T00:52:10.841ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/internet-gateways versions = all
21062025-06-03T00:52:10.841ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/internet-gateways versions = all
21072025-06-03T00:52:10.841ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/internet-gateways/{gateway} versions = all
21082025-06-03T00:52:10.841ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/internet-gateways/{gateway} versions = all
21092025-06-03T00:52:10.841ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/ip-pools versions = all
21102025-06-03T00:52:10.841ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/ip-pools/{pool} versions = all
21112025-06-03T00:52:10.841ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/login/{silo_name}/local versions = all
21122025-06-03T00:52:10.841ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/logout versions = all
21132025-06-03T00:52:10.841ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/me versions = all
21142025-06-03T00:52:10.841ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/me/groups versions = all
21152025-06-03T00:52:10.841ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/me/ssh-keys versions = all
21162025-06-03T00:52:10.841ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/me/ssh-keys versions = all
21172025-06-03T00:52:10.841ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/me/ssh-keys/{ssh_key} versions = all
21182025-06-03T00:52:10.841ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/me/ssh-keys/{ssh_key} versions = all
21192025-06-03T00:52:10.841ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/metrics/{metric_name} versions = all
21202025-06-03T00:52:10.841ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/network-interfaces versions = all
21212025-06-03T00:52:10.841ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/network-interfaces versions = all
21222025-06-03T00:52:10.841ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/network-interfaces/{interface} versions = all
21232025-06-03T00:52:10.841ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/network-interfaces/{interface} versions = all
21242025-06-03T00:52:10.841ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = PUT path = /v1/network-interfaces/{interface} versions = all
21252025-06-03T00:52:10.841ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/ping versions = all
21262025-06-03T00:52:10.841ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/policy versions = all
21272025-06-03T00:52:10.841ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = PUT path = /v1/policy versions = all
21282025-06-03T00:52:10.841ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/projects versions = all
21292025-06-03T00:52:10.841ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/projects versions = all
21302025-06-03T00:52:10.841ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/projects/{project} versions = all
21312025-06-03T00:52:10.841ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/projects/{project} versions = all
21322025-06-03T00:52:10.841ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = PUT path = /v1/projects/{project} versions = all
21332025-06-03T00:52:10.841ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/projects/{project}/policy versions = all
21342025-06-03T00:52:10.841ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = PUT path = /v1/projects/{project}/policy versions = all
21352025-06-03T00:52:10.841ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/snapshots versions = all
21362025-06-03T00:52:10.841ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/snapshots versions = all
21372025-06-03T00:52:10.841ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/snapshots/{snapshot} versions = all
21382025-06-03T00:52:10.841ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/snapshots/{snapshot} versions = all
21392025-06-03T00:52:10.841ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/hardware/disks versions = all
21402025-06-03T00:52:10.841ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/hardware/disks/{disk_id} versions = all
21412025-06-03T00:52:10.841ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/hardware/rack-switch-port/{rack_id}/{switch_location}/{port}/lldp/neighbors versions = all
21422025-06-03T00:52:10.841ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/hardware/racks versions = all
21432025-06-03T00:52:10.841ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/hardware/racks/{rack_id} versions = all
21442025-06-03T00:52:10.841ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/hardware/sleds versions = all
21452025-06-03T00:52:10.841ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/system/hardware/sleds versions = all
21462025-06-03T00:52:10.841ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/hardware/sleds/{sled_id} versions = all
21472025-06-03T00:52:10.841ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/hardware/sleds/{sled_id}/disks versions = all
21482025-06-03T00:52:10.841ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/hardware/sleds/{sled_id}/instances versions = all
21492025-06-03T00:52:10.841ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = PUT path = /v1/system/hardware/sleds/{sled_id}/provision-policy versions = all
21502025-06-03T00:52:10.841ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/hardware/sleds-uninitialized versions = all
21512025-06-03T00:52:10.841ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/hardware/switch-port versions = all
21522025-06-03T00:52:10.841ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/hardware/switch-port/{port}/lldp/config versions = all
21532025-06-03T00:52:10.841ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/system/hardware/switch-port/{port}/lldp/config versions = all
21542025-06-03T00:52:10.841ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/system/hardware/switch-port/{port}/settings versions = all
21552025-06-03T00:52:10.841ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/system/hardware/switch-port/{port}/settings versions = all
21562025-06-03T00:52:10.841ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/hardware/switch-port/{port}/status versions = all
21572025-06-03T00:52:10.841ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/hardware/switches versions = all
21582025-06-03T00:52:10.841ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/hardware/switches/{switch_id} versions = all
21592025-06-03T00:52:10.841ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/identity-providers versions = all
21602025-06-03T00:52:10.841ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/system/identity-providers/local/users versions = all
21612025-06-03T00:52:10.841ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/system/identity-providers/local/users/{user_id} versions = all
21622025-06-03T00:52:10.841ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/system/identity-providers/local/users/{user_id}/set-password versions = all
21632025-06-03T00:52:10.841ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/system/identity-providers/saml versions = all
21642025-06-03T00:52:10.841ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/identity-providers/saml/{provider} versions = all
21652025-06-03T00:52:10.841ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/ip-pools versions = all
21662025-06-03T00:52:10.841ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/system/ip-pools versions = all
21672025-06-03T00:52:10.841ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/system/ip-pools/{pool} versions = all
21682025-06-03T00:52:10.841ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/ip-pools/{pool} versions = all
21692025-06-03T00:52:10.841ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = PUT path = /v1/system/ip-pools/{pool} versions = all
21702025-06-03T00:52:10.841ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/ip-pools/{pool}/ranges versions = all
21712025-06-03T00:52:10.841ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/system/ip-pools/{pool}/ranges/add versions = all
21722025-06-03T00:52:10.841ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/system/ip-pools/{pool}/ranges/remove versions = all
21732025-06-03T00:52:10.841ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/ip-pools/{pool}/silos versions = all
21742025-06-03T00:52:10.841ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/system/ip-pools/{pool}/silos versions = all
21752025-06-03T00:52:10.841ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/system/ip-pools/{pool}/silos/{silo} versions = all
21762025-06-03T00:52:10.841ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = PUT path = /v1/system/ip-pools/{pool}/silos/{silo} versions = all
21772025-06-03T00:52:10.841ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/ip-pools/{pool}/utilization versions = all
21782025-06-03T00:52:10.841ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/ip-pools-service versions = all
21792025-06-03T00:52:10.841ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/ip-pools-service/ranges versions = all
21802025-06-03T00:52:10.842ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/system/ip-pools-service/ranges/add versions = all
21812025-06-03T00:52:10.842ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/system/ip-pools-service/ranges/remove versions = all
21822025-06-03T00:52:10.842ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/metrics/{metric_name} versions = all
21832025-06-03T00:52:10.842ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/networking/address-lot versions = all
21842025-06-03T00:52:10.842ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/system/networking/address-lot versions = all
21852025-06-03T00:52:10.842ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/system/networking/address-lot/{address_lot} versions = all
21862025-06-03T00:52:10.842ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/networking/address-lot/{address_lot}/blocks versions = all
21872025-06-03T00:52:10.842ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/networking/allow-list versions = all
21882025-06-03T00:52:10.842ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = PUT path = /v1/system/networking/allow-list versions = all
21892025-06-03T00:52:10.842ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/system/networking/bfd-disable versions = all
21902025-06-03T00:52:10.842ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/system/networking/bfd-enable versions = all
21912025-06-03T00:52:10.842ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/networking/bfd-status versions = all
21922025-06-03T00:52:10.842ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/system/networking/bgp versions = all
21932025-06-03T00:52:10.842ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/networking/bgp versions = all
21942025-06-03T00:52:10.842ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/system/networking/bgp versions = all
21952025-06-03T00:52:10.842ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/networking/bgp-announce-set versions = all
21962025-06-03T00:52:10.842ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = PUT path = /v1/system/networking/bgp-announce-set versions = all
21972025-06-03T00:52:10.842ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/system/networking/bgp-announce-set/{announce_set} versions = all
21982025-06-03T00:52:10.842ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/networking/bgp-announce-set/{announce_set}/announcement versions = all
21992025-06-03T00:52:10.842ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/networking/bgp-exported versions = all
22002025-06-03T00:52:10.842ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/networking/bgp-message-history versions = all
22012025-06-03T00:52:10.842ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/networking/bgp-routes-ipv4 versions = all
22022025-06-03T00:52:10.842ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/networking/bgp-status versions = all
22032025-06-03T00:52:10.842ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/networking/loopback-address versions = all
22042025-06-03T00:52:10.842ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/system/networking/loopback-address versions = all
22052025-06-03T00:52:10.842ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/system/networking/loopback-address/{rack_id}/{switch_location}/{address}/{subnet_mask} versions = all
22062025-06-03T00:52:10.842ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/system/networking/switch-port-settings versions = all
22072025-06-03T00:52:10.842ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/networking/switch-port-settings versions = all
22082025-06-03T00:52:10.842ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/system/networking/switch-port-settings versions = all
22092025-06-03T00:52:10.842ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/networking/switch-port-settings/{port} versions = all
22102025-06-03T00:52:10.842ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/policy versions = all
22112025-06-03T00:52:10.842ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = PUT path = /v1/system/policy versions = all
22122025-06-03T00:52:10.842ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/roles versions = all
22132025-06-03T00:52:10.842ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/roles/{role_name} versions = all
22142025-06-03T00:52:10.842ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/silo-quotas versions = all
22152025-06-03T00:52:10.842ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/silos versions = all
22162025-06-03T00:52:10.842ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/system/silos versions = all
22172025-06-03T00:52:10.842ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/system/silos/{silo} versions = all
22182025-06-03T00:52:10.842ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/silos/{silo} versions = all
22192025-06-03T00:52:10.842ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/silos/{silo}/ip-pools versions = all
22202025-06-03T00:52:10.842ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/silos/{silo}/policy versions = all
22212025-06-03T00:52:10.842ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = PUT path = /v1/system/silos/{silo}/policy versions = all
22222025-06-03T00:52:10.842ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/silos/{silo}/quotas versions = all
22232025-06-03T00:52:10.842ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = PUT path = /v1/system/silos/{silo}/quotas versions = all
22242025-06-03T00:52:10.842ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/system/timeseries/query versions = all
22252025-06-03T00:52:10.842ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/timeseries/schemas versions = all
22262025-06-03T00:52:10.842ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = PUT path = /v1/system/update/repository versions = all
22272025-06-03T00:52:10.842ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/update/repository/{system_version} versions = all
22282025-06-03T00:52:10.842ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/update/target-release versions = all
22292025-06-03T00:52:10.842ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = PUT path = /v1/system/update/target-release versions = all
22302025-06-03T00:52:10.842ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/users versions = all
22312025-06-03T00:52:10.842ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/users/{user_id} versions = all
22322025-06-03T00:52:10.842ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/users-builtin versions = all
22332025-06-03T00:52:10.842ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/users-builtin/{user} versions = all
22342025-06-03T00:52:10.842ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/utilization/silos versions = all
22352025-06-03T00:52:10.842ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/utilization/silos/{silo} versions = all
22362025-06-03T00:52:10.842ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/timeseries/query versions = all
22372025-06-03T00:52:10.842ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/users versions = all
22382025-06-03T00:52:10.842ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/utilization versions = all
22392025-06-03T00:52:10.842ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/vpc-firewall-rules versions = all
22402025-06-03T00:52:10.842ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = PUT path = /v1/vpc-firewall-rules versions = all
22412025-06-03T00:52:10.842ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/vpc-router-routes versions = all
22422025-06-03T00:52:10.842ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/vpc-router-routes versions = all
22432025-06-03T00:52:10.842ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/vpc-router-routes/{route} versions = all
22442025-06-03T00:52:10.842ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/vpc-router-routes/{route} versions = all
22452025-06-03T00:52:10.842ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = PUT path = /v1/vpc-router-routes/{route} versions = all
22462025-06-03T00:52:10.842ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/vpc-routers versions = all
22472025-06-03T00:52:10.842ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/vpc-routers versions = all
22482025-06-03T00:52:10.842ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/vpc-routers/{router} versions = all
22492025-06-03T00:52:10.842ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/vpc-routers/{router} versions = all
22502025-06-03T00:52:10.842ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = PUT path = /v1/vpc-routers/{router} versions = all
22512025-06-03T00:52:10.842ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/vpc-subnets versions = all
22522025-06-03T00:52:10.842ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/vpc-subnets versions = all
22532025-06-03T00:52:10.842ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/vpc-subnets/{subnet} versions = all
22542025-06-03T00:52:10.842ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/vpc-subnets/{subnet} versions = all
22552025-06-03T00:52:10.842ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = PUT path = /v1/vpc-subnets/{subnet} versions = all
22562025-06-03T00:52:10.842ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/vpc-subnets/{subnet}/network-interfaces versions = all
22572025-06-03T00:52:10.842ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/vpcs versions = all
22582025-06-03T00:52:10.842ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/vpcs versions = all
22592025-06-03T00:52:10.842ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/vpcs/{vpc} versions = all
22602025-06-03T00:52:10.842ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/vpcs/{vpc} versions = all
22612025-06-03T00:52:10.842ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = PUT path = /v1/vpcs/{vpc} versions = all
22622025-06-03T00:52:10.842ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/webhook-receivers versions = all
22632025-06-03T00:52:10.842ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = PUT path = /v1/webhook-receivers/{receiver} versions = all
22642025-06-03T00:52:10.842ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/webhook-secrets versions = all
22652025-06-03T00:52:10.842ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/webhook-secrets versions = all
22662025-06-03T00:52:10.842ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/webhook-secrets/{secret_id} versions = all
22672025-06-03T00:52:10.843ZINFOnexus (dropshot_external): listening file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:328 local_addr = 172.30.2.5:443
22682025-06-03T00:52:10.843ZDEBGnexus (dropshot_external): successfully registered DTrace USDT probes local_addr = 172.30.2.5:443
22692025-06-03T00:52:10.843ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
22702025-06-03T00:52:10.844ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
22712025-06-03T00:52:10.845ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 9e79f9b1-a4de-4f9d-a901-16cc4f79e15d dns_zone_name = control-plane.oxide.internal found_so_far = 59 version = 1
22722025-06-03T00:52:10.845ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
22732025-06-03T00:52:10.845ZINFOnexus (BackgroundTasks): found latest generation (first find) background_task = dns_config_internal dns_group = internal file = nexus/src/app/background/tasks/dns_config.rs:87 generation = 1
22742025-06-03T00:52:10.845ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 338.575853ms iteration = 2 status = {"generation":1}
22752025-06-03T00:52:10.845ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 4 reason = Dependency
22762025-06-03T00:52:10.845ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
22772025-06-03T00:52:10.845ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 3ef110b4-9400-4c37-8d7f-5552c8a94db0 dns_zone_name = oxide.test found_so_far = 0 version = 2
22782025-06-03T00:52:10.845ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
22792025-06-03T00:52:10.846ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22802025-06-03T00:52:10.846ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
22812025-06-03T00:52:10.847ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
22822025-06-03T00:52:10.847ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22832025-06-03T00:52:10.847ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
22842025-06-03T00:52:10.848ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
22852025-06-03T00:52:10.849ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22862025-06-03T00:52:10.850ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
22872025-06-03T00:52:10.850ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
22882025-06-03T00:52:10.850ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
22892025-06-03T00:52:10.851ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
22902025-06-03T00:52:10.851ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
22912025-06-03T00:52:10.852ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
22922025-06-03T00:52:10.852ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
22932025-06-03T00:52:10.853ZDEBGnexus (BackgroundTasks): client request SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
22942025-06-03T00:52:10.853ZDEBGnexus (BackgroundTasks): client response SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "d771d91a-2293-46f1-b0fa-9e68f1f6886e", "date": "Tue, 03 Jun 2025 00:52:10 GMT"} })
22952025-06-03T00:52:10.853ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
22962025-06-03T00:52:10.853ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
22972025-06-03T00:52:10.854ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = / versions = all
22982025-06-03T00:52:10.854ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /access versions = all
22992025-06-03T00:52:10.854ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /assets/{path} versions = all
23002025-06-03T00:52:10.854ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /device/auth versions = all
23012025-06-03T00:52:10.854ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /device/confirm versions = all
23022025-06-03T00:52:10.854ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /device/success versions = all
23032025-06-03T00:52:10.854ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /device/token versions = all
23042025-06-03T00:52:10.854ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /device/verify versions = all
23052025-06-03T00:52:10.854ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /experimental/v1/probes versions = all
23062025-06-03T00:52:10.854ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /experimental/v1/probes versions = all
23072025-06-03T00:52:10.854ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /experimental/v1/probes/{probe} versions = all
23082025-06-03T00:52:10.854ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /experimental/v1/probes/{probe} versions = all
23092025-06-03T00:52:10.854ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /experimental/v1/system/support-bundles versions = all
23102025-06-03T00:52:10.855ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /experimental/v1/system/support-bundles versions = all
23112025-06-03T00:52:10.855ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /experimental/v1/system/support-bundles/{bundle_id} versions = all
23122025-06-03T00:52:10.855ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /experimental/v1/system/support-bundles/{bundle_id} versions = all
23132025-06-03T00:52:10.855ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /experimental/v1/system/support-bundles/{bundle_id}/download versions = all
23142025-06-03T00:52:10.855ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = HEAD path = /experimental/v1/system/support-bundles/{bundle_id}/download versions = all
23152025-06-03T00:52:10.855ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /experimental/v1/system/support-bundles/{bundle_id}/download/{file} versions = all
23162025-06-03T00:52:10.855ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = HEAD path = /experimental/v1/system/support-bundles/{bundle_id}/download/{file} versions = all
23172025-06-03T00:52:10.855ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /experimental/v1/system/support-bundles/{bundle_id}/index versions = all
23182025-06-03T00:52:10.855ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /images versions = all
23192025-06-03T00:52:10.855ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /login versions = all
23202025-06-03T00:52:10.855ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /login/{silo_name}/local versions = all
23212025-06-03T00:52:10.855ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /login/{silo_name}/saml/{provider_name} versions = all
23222025-06-03T00:52:10.855ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /login/{silo_name}/saml/{provider_name} versions = all
23232025-06-03T00:52:10.855ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /login/{silo_name}/saml/{provider_name}/redirect versions = all
23242025-06-03T00:52:10.855ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /lookup/{path} versions = all
23252025-06-03T00:52:10.855ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /projects/{path} versions = all
23262025-06-03T00:52:10.855ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /projects-new versions = all
23272025-06-03T00:52:10.855ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /settings/{path} versions = all
23282025-06-03T00:52:10.855ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /system/{path} versions = all
23292025-06-03T00:52:10.855ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /utilization versions = all
23302025-06-03T00:52:10.855ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/affinity-groups versions = all
23312025-06-03T00:52:10.855ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/affinity-groups versions = all
23322025-06-03T00:52:10.855ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/affinity-groups/{affinity_group} versions = all
23332025-06-03T00:52:10.855ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/affinity-groups/{affinity_group} versions = all
23342025-06-03T00:52:10.855ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = PUT path = /v1/affinity-groups/{affinity_group} versions = all
23352025-06-03T00:52:10.855ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/affinity-groups/{affinity_group}/members versions = all
23362025-06-03T00:52:10.855ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/affinity-groups/{affinity_group}/members/instance/{instance} versions = all
23372025-06-03T00:52:10.855ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/affinity-groups/{affinity_group}/members/instance/{instance} versions = all
23382025-06-03T00:52:10.855ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/affinity-groups/{affinity_group}/members/instance/{instance} versions = all
23392025-06-03T00:52:10.855ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/alert-classes versions = all
23402025-06-03T00:52:10.855ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/alert-receivers versions = all
23412025-06-03T00:52:10.855ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/alert-receivers/{receiver} versions = all
23422025-06-03T00:52:10.855ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/alert-receivers/{receiver} versions = all
23432025-06-03T00:52:10.855ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/alert-receivers/{receiver}/deliveries versions = all
23442025-06-03T00:52:10.855ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/alert-receivers/{receiver}/probe versions = all
23452025-06-03T00:52:10.855ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/alert-receivers/{receiver}/subscriptions versions = all
23462025-06-03T00:52:10.855ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/alert-receivers/{receiver}/subscriptions/{subscription} versions = all
23472025-06-03T00:52:10.855ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/alerts/{alert_id}/resend versions = all
23482025-06-03T00:52:10.855ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/anti-affinity-groups versions = all
23492025-06-03T00:52:10.855ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/anti-affinity-groups versions = all
23502025-06-03T00:52:10.855ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/anti-affinity-groups/{anti_affinity_group} versions = all
23512025-06-03T00:52:10.855ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/anti-affinity-groups/{anti_affinity_group} versions = all
23522025-06-03T00:52:10.855ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = PUT path = /v1/anti-affinity-groups/{anti_affinity_group} versions = all
23532025-06-03T00:52:10.855ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/anti-affinity-groups/{anti_affinity_group}/members versions = all
23542025-06-03T00:52:10.855ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/anti-affinity-groups/{anti_affinity_group}/members/instance/{instance} versions = all
23552025-06-03T00:52:10.855ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/anti-affinity-groups/{anti_affinity_group}/members/instance/{instance} versions = all
23562025-06-03T00:52:10.855ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/anti-affinity-groups/{anti_affinity_group}/members/instance/{instance} versions = all
23572025-06-03T00:52:10.855ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/certificates versions = all
23582025-06-03T00:52:10.855ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/certificates versions = all
23592025-06-03T00:52:10.855ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/certificates/{certificate} versions = all
23602025-06-03T00:52:10.855ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/certificates/{certificate} versions = all
23612025-06-03T00:52:10.855ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/disks versions = all
23622025-06-03T00:52:10.855ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/disks versions = all
23632025-06-03T00:52:10.855ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/disks/{disk} versions = all
23642025-06-03T00:52:10.855ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/disks/{disk} versions = all
23652025-06-03T00:52:10.855ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/disks/{disk}/bulk-write versions = all
23662025-06-03T00:52:10.855ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/disks/{disk}/bulk-write-start versions = all
23672025-06-03T00:52:10.855ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/disks/{disk}/bulk-write-stop versions = all
23682025-06-03T00:52:10.855ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/disks/{disk}/finalize versions = all
23692025-06-03T00:52:10.855ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/disks/{disk}/metrics/{metric} versions = all
23702025-06-03T00:52:10.855ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/floating-ips versions = all
23712025-06-03T00:52:10.855ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/floating-ips versions = all
23722025-06-03T00:52:10.855ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/floating-ips/{floating_ip} versions = all
23732025-06-03T00:52:10.855ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/floating-ips/{floating_ip} versions = all
23742025-06-03T00:52:10.855ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = PUT path = /v1/floating-ips/{floating_ip} versions = all
23752025-06-03T00:52:10.855ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/floating-ips/{floating_ip}/attach versions = all
23762025-06-03T00:52:10.855ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/floating-ips/{floating_ip}/detach versions = all
23772025-06-03T00:52:10.855ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/groups versions = all
23782025-06-03T00:52:10.855ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/groups/{group_id} versions = all
23792025-06-03T00:52:10.855ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/images versions = all
23802025-06-03T00:52:10.855ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/images versions = all
23812025-06-03T00:52:10.855ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/images/{image} versions = all
23822025-06-03T00:52:10.855ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/images/{image} versions = all
23832025-06-03T00:52:10.855ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/images/{image}/demote versions = all
23842025-06-03T00:52:10.855ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/images/{image}/promote versions = all
23852025-06-03T00:52:10.855ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/instances versions = all
23862025-06-03T00:52:10.855ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/instances versions = all
23872025-06-03T00:52:10.855ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/instances/{instance} versions = all
23882025-06-03T00:52:10.855ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/instances/{instance} versions = all
23892025-06-03T00:52:10.855ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = PUT path = /v1/instances/{instance} versions = all
23902025-06-03T00:52:10.855ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/instances/{instance}/affinity-groups versions = all
23912025-06-03T00:52:10.855ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/instances/{instance}/anti-affinity-groups versions = all
23922025-06-03T00:52:10.855ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/instances/{instance}/disks versions = all
23932025-06-03T00:52:10.855ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/instances/{instance}/disks/attach versions = all
23942025-06-03T00:52:10.856ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/instances/{instance}/disks/detach versions = all
23952025-06-03T00:52:10.856ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/instances/{instance}/external-ips versions = all
23962025-06-03T00:52:10.856ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/instances/{instance}/external-ips/ephemeral versions = all
23972025-06-03T00:52:10.856ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/instances/{instance}/external-ips/ephemeral versions = all
23982025-06-03T00:52:10.856ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/instances/{instance}/reboot versions = all
23992025-06-03T00:52:10.856ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/instances/{instance}/serial-console versions = all
24002025-06-03T00:52:10.856ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/instances/{instance}/serial-console/stream versions = all
24012025-06-03T00:52:10.856ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/instances/{instance}/ssh-public-keys versions = all
24022025-06-03T00:52:10.856ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/instances/{instance}/start versions = all
24032025-06-03T00:52:10.856ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/instances/{instance}/stop versions = all
24042025-06-03T00:52:10.856ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/internet-gateway-ip-addresses versions = all
24052025-06-03T00:52:10.856ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/internet-gateway-ip-addresses versions = all
24062025-06-03T00:52:10.856ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/internet-gateway-ip-addresses/{address} versions = all
24072025-06-03T00:52:10.856ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/internet-gateway-ip-pools versions = all
24082025-06-03T00:52:10.856ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/internet-gateway-ip-pools versions = all
24092025-06-03T00:52:10.856ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/internet-gateway-ip-pools/{pool} versions = all
24102025-06-03T00:52:10.856ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/internet-gateways versions = all
24112025-06-03T00:52:10.856ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/internet-gateways versions = all
24122025-06-03T00:52:10.856ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/internet-gateways/{gateway} versions = all
24132025-06-03T00:52:10.856ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/internet-gateways/{gateway} versions = all
24142025-06-03T00:52:10.856ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/ip-pools versions = all
24152025-06-03T00:52:10.856ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/ip-pools/{pool} versions = all
24162025-06-03T00:52:10.856ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/login/{silo_name}/local versions = all
24172025-06-03T00:52:10.856ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/logout versions = all
24182025-06-03T00:52:10.856ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/me versions = all
24192025-06-03T00:52:10.856ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/me/groups versions = all
24202025-06-03T00:52:10.856ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/me/ssh-keys versions = all
24212025-06-03T00:52:10.856ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/me/ssh-keys versions = all
24222025-06-03T00:52:10.856ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/me/ssh-keys/{ssh_key} versions = all
24232025-06-03T00:52:10.856ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/me/ssh-keys/{ssh_key} versions = all
24242025-06-03T00:52:10.856ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/metrics/{metric_name} versions = all
24252025-06-03T00:52:10.856ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/network-interfaces versions = all
24262025-06-03T00:52:10.856ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/network-interfaces versions = all
24272025-06-03T00:52:10.856ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/network-interfaces/{interface} versions = all
24282025-06-03T00:52:10.856ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/network-interfaces/{interface} versions = all
24292025-06-03T00:52:10.856ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = PUT path = /v1/network-interfaces/{interface} versions = all
24302025-06-03T00:52:10.856ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/ping versions = all
24312025-06-03T00:52:10.856ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/policy versions = all
24322025-06-03T00:52:10.856ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = PUT path = /v1/policy versions = all
24332025-06-03T00:52:10.856ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/projects versions = all
24342025-06-03T00:52:10.856ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/projects versions = all
24352025-06-03T00:52:10.856ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/projects/{project} versions = all
24362025-06-03T00:52:10.856ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/projects/{project} versions = all
24372025-06-03T00:52:10.856ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = PUT path = /v1/projects/{project} versions = all
24382025-06-03T00:52:10.856ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/projects/{project}/policy versions = all
24392025-06-03T00:52:10.856ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = PUT path = /v1/projects/{project}/policy versions = all
24402025-06-03T00:52:10.856ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/snapshots versions = all
24412025-06-03T00:52:10.856ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/snapshots versions = all
24422025-06-03T00:52:10.856ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/snapshots/{snapshot} versions = all
24432025-06-03T00:52:10.856ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/snapshots/{snapshot} versions = all
24442025-06-03T00:52:10.856ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/hardware/disks versions = all
24452025-06-03T00:52:10.856ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/hardware/disks/{disk_id} versions = all
24462025-06-03T00:52:10.856ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/hardware/rack-switch-port/{rack_id}/{switch_location}/{port}/lldp/neighbors versions = all
24472025-06-03T00:52:10.856ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/hardware/racks versions = all
24482025-06-03T00:52:10.856ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/hardware/racks/{rack_id} versions = all
24492025-06-03T00:52:10.856ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/hardware/sleds versions = all
24502025-06-03T00:52:10.856ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/system/hardware/sleds versions = all
24512025-06-03T00:52:10.856ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/hardware/sleds/{sled_id} versions = all
24522025-06-03T00:52:10.856ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/hardware/sleds/{sled_id}/disks versions = all
24532025-06-03T00:52:10.856ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/hardware/sleds/{sled_id}/instances versions = all
24542025-06-03T00:52:10.856ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = PUT path = /v1/system/hardware/sleds/{sled_id}/provision-policy versions = all
24552025-06-03T00:52:10.856ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/hardware/sleds-uninitialized versions = all
24562025-06-03T00:52:10.856ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/hardware/switch-port versions = all
24572025-06-03T00:52:10.856ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/hardware/switch-port/{port}/lldp/config versions = all
24582025-06-03T00:52:10.856ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/system/hardware/switch-port/{port}/lldp/config versions = all
24592025-06-03T00:52:10.856ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/system/hardware/switch-port/{port}/settings versions = all
24602025-06-03T00:52:10.856ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/system/hardware/switch-port/{port}/settings versions = all
24612025-06-03T00:52:10.856ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/hardware/switch-port/{port}/status versions = all
24622025-06-03T00:52:10.856ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/hardware/switches versions = all
24632025-06-03T00:52:10.856ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/hardware/switches/{switch_id} versions = all
24642025-06-03T00:52:10.856ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/identity-providers versions = all
24652025-06-03T00:52:10.856ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/system/identity-providers/local/users versions = all
24662025-06-03T00:52:10.856ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/system/identity-providers/local/users/{user_id} versions = all
24672025-06-03T00:52:10.856ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/system/identity-providers/local/users/{user_id}/set-password versions = all
24682025-06-03T00:52:10.856ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/system/identity-providers/saml versions = all
24692025-06-03T00:52:10.856ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/identity-providers/saml/{provider} versions = all
24702025-06-03T00:52:10.856ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/ip-pools versions = all
24712025-06-03T00:52:10.856ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/system/ip-pools versions = all
24722025-06-03T00:52:10.856ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/system/ip-pools/{pool} versions = all
24732025-06-03T00:52:10.856ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/ip-pools/{pool} versions = all
24742025-06-03T00:52:10.856ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = PUT path = /v1/system/ip-pools/{pool} versions = all
24752025-06-03T00:52:10.856ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/ip-pools/{pool}/ranges versions = all
24762025-06-03T00:52:10.856ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/system/ip-pools/{pool}/ranges/add versions = all
24772025-06-03T00:52:10.856ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/system/ip-pools/{pool}/ranges/remove versions = all
24782025-06-03T00:52:10.857ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/ip-pools/{pool}/silos versions = all
24792025-06-03T00:52:10.857ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/system/ip-pools/{pool}/silos versions = all
24802025-06-03T00:52:10.857ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/system/ip-pools/{pool}/silos/{silo} versions = all
24812025-06-03T00:52:10.857ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = PUT path = /v1/system/ip-pools/{pool}/silos/{silo} versions = all
24822025-06-03T00:52:10.857ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/ip-pools/{pool}/utilization versions = all
24832025-06-03T00:52:10.857ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/ip-pools-service versions = all
24842025-06-03T00:52:10.857ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/ip-pools-service/ranges versions = all
24852025-06-03T00:52:10.857ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/system/ip-pools-service/ranges/add versions = all
24862025-06-03T00:52:10.857ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/system/ip-pools-service/ranges/remove versions = all
24872025-06-03T00:52:10.857ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/metrics/{metric_name} versions = all
24882025-06-03T00:52:10.857ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/networking/address-lot versions = all
24892025-06-03T00:52:10.857ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/system/networking/address-lot versions = all
24902025-06-03T00:52:10.857ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/system/networking/address-lot/{address_lot} versions = all
24912025-06-03T00:52:10.857ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/networking/address-lot/{address_lot}/blocks versions = all
24922025-06-03T00:52:10.857ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/networking/allow-list versions = all
24932025-06-03T00:52:10.857ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = PUT path = /v1/system/networking/allow-list versions = all
24942025-06-03T00:52:10.857ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/system/networking/bfd-disable versions = all
24952025-06-03T00:52:10.857ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/system/networking/bfd-enable versions = all
24962025-06-03T00:52:10.857ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/networking/bfd-status versions = all
24972025-06-03T00:52:10.857ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/system/networking/bgp versions = all
24982025-06-03T00:52:10.857ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/networking/bgp versions = all
24992025-06-03T00:52:10.857ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/system/networking/bgp versions = all
25002025-06-03T00:52:10.857ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/networking/bgp-announce-set versions = all
25012025-06-03T00:52:10.857ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = PUT path = /v1/system/networking/bgp-announce-set versions = all
25022025-06-03T00:52:10.857ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/system/networking/bgp-announce-set/{announce_set} versions = all
25032025-06-03T00:52:10.857ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/networking/bgp-announce-set/{announce_set}/announcement versions = all
25042025-06-03T00:52:10.857ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/networking/bgp-exported versions = all
25052025-06-03T00:52:10.857ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/networking/bgp-message-history versions = all
25062025-06-03T00:52:10.857ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/networking/bgp-routes-ipv4 versions = all
25072025-06-03T00:52:10.857ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/networking/bgp-status versions = all
25082025-06-03T00:52:10.857ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/networking/loopback-address versions = all
25092025-06-03T00:52:10.857ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/system/networking/loopback-address versions = all
25102025-06-03T00:52:10.857ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/system/networking/loopback-address/{rack_id}/{switch_location}/{address}/{subnet_mask} versions = all
25112025-06-03T00:52:10.857ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/system/networking/switch-port-settings versions = all
25122025-06-03T00:52:10.857ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/networking/switch-port-settings versions = all
25132025-06-03T00:52:10.857ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/system/networking/switch-port-settings versions = all
25142025-06-03T00:52:10.857ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/networking/switch-port-settings/{port} versions = all
25152025-06-03T00:52:10.857ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/policy versions = all
25162025-06-03T00:52:10.857ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = PUT path = /v1/system/policy versions = all
25172025-06-03T00:52:10.857ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/roles versions = all
25182025-06-03T00:52:10.857ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/roles/{role_name} versions = all
25192025-06-03T00:52:10.857ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/silo-quotas versions = all
25202025-06-03T00:52:10.858ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/silos versions = all
25212025-06-03T00:52:10.858ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/system/silos versions = all
25222025-06-03T00:52:10.858ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/system/silos/{silo} versions = all
25232025-06-03T00:52:10.858ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/silos/{silo} versions = all
25242025-06-03T00:52:10.858ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/silos/{silo}/ip-pools versions = all
25252025-06-03T00:52:10.858ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/silos/{silo}/policy versions = all
25262025-06-03T00:52:10.858ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = PUT path = /v1/system/silos/{silo}/policy versions = all
25272025-06-03T00:52:10.858ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/silos/{silo}/quotas versions = all
25282025-06-03T00:52:10.858ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = PUT path = /v1/system/silos/{silo}/quotas versions = all
25292025-06-03T00:52:10.858ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/system/timeseries/query versions = all
25302025-06-03T00:52:10.858ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/timeseries/schemas versions = all
25312025-06-03T00:52:10.858ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = PUT path = /v1/system/update/repository versions = all
25322025-06-03T00:52:10.858ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/update/repository/{system_version} versions = all
25332025-06-03T00:52:10.858ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/update/target-release versions = all
25342025-06-03T00:52:10.858ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = PUT path = /v1/system/update/target-release versions = all
25352025-06-03T00:52:10.858ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/users versions = all
25362025-06-03T00:52:10.858ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/users/{user_id} versions = all
25372025-06-03T00:52:10.858ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/users-builtin versions = all
25382025-06-03T00:52:10.858ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/users-builtin/{user} versions = all
25392025-06-03T00:52:10.858ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/utilization/silos versions = all
25402025-06-03T00:52:10.858ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/utilization/silos/{silo} versions = all
25412025-06-03T00:52:10.858ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/timeseries/query versions = all
25422025-06-03T00:52:10.858ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/users versions = all
25432025-06-03T00:52:10.858ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/utilization versions = all
25442025-06-03T00:52:10.858ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/vpc-firewall-rules versions = all
25452025-06-03T00:52:10.858ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = PUT path = /v1/vpc-firewall-rules versions = all
25462025-06-03T00:52:10.858ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/vpc-router-routes versions = all
25472025-06-03T00:52:10.858ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/vpc-router-routes versions = all
25482025-06-03T00:52:10.858ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/vpc-router-routes/{route} versions = all
25492025-06-03T00:52:10.858ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/vpc-router-routes/{route} versions = all
25502025-06-03T00:52:10.858ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = PUT path = /v1/vpc-router-routes/{route} versions = all
25512025-06-03T00:52:10.858ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/vpc-routers versions = all
25522025-06-03T00:52:10.858ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/vpc-routers versions = all
25532025-06-03T00:52:10.858ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/vpc-routers/{router} versions = all
25542025-06-03T00:52:10.858ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/vpc-routers/{router} versions = all
25552025-06-03T00:52:10.858ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = PUT path = /v1/vpc-routers/{router} versions = all
25562025-06-03T00:52:10.858ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/vpc-subnets versions = all
25572025-06-03T00:52:10.858ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/vpc-subnets versions = all
25582025-06-03T00:52:10.858ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/vpc-subnets/{subnet} versions = all
25592025-06-03T00:52:10.858ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/vpc-subnets/{subnet} versions = all
25602025-06-03T00:52:10.858ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = PUT path = /v1/vpc-subnets/{subnet} versions = all
25612025-06-03T00:52:10.858ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/vpc-subnets/{subnet}/network-interfaces versions = all
25622025-06-03T00:52:10.858ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/vpcs versions = all
25632025-06-03T00:52:10.858ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/vpcs versions = all
25642025-06-03T00:52:10.858ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/vpcs/{vpc} versions = all
25652025-06-03T00:52:10.858ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/vpcs/{vpc} versions = all
25662025-06-03T00:52:10.858ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = PUT path = /v1/vpcs/{vpc} versions = all
25672025-06-03T00:52:10.858ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/webhook-receivers versions = all
25682025-06-03T00:52:10.858ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = PUT path = /v1/webhook-receivers/{receiver} versions = all
25692025-06-03T00:52:10.858ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/webhook-secrets versions = all
25702025-06-03T00:52:10.858ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/webhook-secrets versions = all
25712025-06-03T00:52:10.858ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/webhook-secrets/{secret_id} versions = all
25722025-06-03T00:52:10.858ZINFOnexus (dropshot_external_techport): listening file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:328 local_addr = [fd00:1122:3344:101::a]:12228
25732025-06-03T00:52:10.858ZDEBGnexus (dropshot_external_techport): successfully registered DTrace USDT probes local_addr = [fd00:1122:3344:101::a]:12228
25742025-06-03T00:52:10.858ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
25752025-06-03T00:52:10.858ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
25762025-06-03T00:52:10.860ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 3ef110b4-9400-4c37-8d7f-5552c8a94db0 dns_zone_name = oxide.test found_so_far = 4 version = 2
25772025-06-03T00:52:10.860ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
25782025-06-03T00:52:10.860ZINFOnexus (BackgroundTasks): found latest generation (first find) background_task = dns_config_external dns_group = external file = nexus/src/app/background/tasks/dns_config.rs:87 generation = 2
25792025-06-03T00:52:10.860ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 351.596778ms iteration = 2 status = {"generation":2}
25802025-06-03T00:52:10.860ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 4 reason = Dependency
25812025-06-03T00:52:10.862ZDEBGnexus (nexus-producer-server): registered DTrace probes
25822025-06-03T00:52:10.869ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 361.445254ms iteration = 3 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
25832025-06-03T00:52:10.871ZDEBGnexus (dropshot): registered endpoint local_addr = [fd00:1122:3344:101::a]:63576 method = GET path = /{producer_id} versions = all
25842025-06-03T00:52:10.871ZINFOnexus (dropshot): listening file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:328 local_addr = [fd00:1122:3344:101::a]:63576
25852025-06-03T00:52:10.871ZDEBGnexus (dropshot): successfully registered DTrace USDT probes local_addr = [fd00:1122:3344:101::a]:63576
25862025-06-03T00:52:10.871ZDEBGnexus (nexus-producer-server): Requested any available port, Dropshot server has been bound to [fd00:1122:3344:101::a]:63576
25872025-06-03T00:52:10.871ZDEBGnexus (nexus-producer-server): Nexus IP provided explicitly, will use it addr = [fd00:1122:3344:101::a]:12221
25882025-06-03T00:52:10.871ZDEBGnexus (nexus-producer-server): starting producer registration task
25892025-06-03T00:52:10.873ZINFOnexus (nexus-producer-server): starting oximeter metric producer server address = [fd00:1122:3344:101::a]:63576 file = oximeter/producer/src/lib.rs:279 interval = 10s producer_id = a2861392-5fd2-462b-84e5-6f36c716263a
25902025-06-03T00:52:10.873ZDEBGnexus (producer-registration-task): registering as oximeter producer with Nexus registration_count = 0
25912025-06-03T00:52:10.873ZDEBGnexus (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::a]:12221
25922025-06-03T00:52:10.878ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
25932025-06-03T00:52:10.878ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
25942025-06-03T00:52:10.880ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 372.27376ms iteration = 3 status = null
25952025-06-03T00:52:10.888ZINFOnexus (BackgroundTasks): instance reincarnation completed successfully background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:113 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0
25962025-06-03T00:52:10.888ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 379.602897ms iteration = 2 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
25972025-06-03T00:52:10.969ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
25982025-06-03T00:52:10.969ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
25992025-06-03T00:52:10.974ZINFOnexus (BackgroundTasks): VPC route manager sled afe89ed7-19b7-479b-81eb-05c988ebc66d push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
26002025-06-03T00:52:10.974ZDEBGnexus (BackgroundTasks): client request SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
26012025-06-03T00:52:10.974ZDEBGnexus (BackgroundTasks): client response SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "0c872ab6-aff3-43d6-95a1-43e6165261f8", "date": "Tue, 03 Jun 2025 00:52:10 GMT"} })
26022025-06-03T00:52:10.976ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 467.756095ms iteration = 3 status = {}
26032025-06-03T00:52:10.988ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
26042025-06-03T00:52:11.013ZDEBGnexus (producer-registration-task): client request body = Some(Body) method = POST uri = http://[fd00:1122:3344:101::a]:12221/metrics/producers
26052025-06-03T00:52:11.013ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:1025 local_addr = [fd00:1122:3344:101::a]:12221 remote_addr = [fd00:1122:3344:101::a]:47730
26062025-06-03T00:52:11.028ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = POST remote_addr = [fd00:1122:3344:101::a]:47730 req_id = d53a649f-f164-42c5-9eb3-a2ece61781d6 roles = RoleSet { roles: {} } uri = /metrics/producers
26072025-06-03T00:52:11.028ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = POST remote_addr = [fd00:1122:3344:101::a]:47730 req_id = d53a649f-f164-42c5-9eb3-a2ece61781d6 resource = Database result = Ok(()) uri = /metrics/producers
26082025-06-03T00:52:11.076ZINFOnexus: assigned collector to new producer collector_id = 292a19e1-4305-4e49-b11e-08ac74714bf3 file = nexus/src/app/oximeter.rs:76 producer_id = a2861392-5fd2-462b-84e5-6f36c716263a
26092025-06-03T00:52:11.076ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 63304 local_addr = [fd00:1122:3344:101::a]:12221 method = POST remote_addr = [fd00:1122:3344:101::a]:47730 req_id = d53a649f-f164-42c5-9eb3-a2ece61781d6 response_code = 201 uri = /metrics/producers
26102025-06-03T00:52:11.076ZDEBGnexus (producer-registration-task): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::a]:12221/metrics/producers", status: 201, headers: {"content-type": "application/json", "x-request-id": "d53a649f-f164-42c5-9eb3-a2ece61781d6", "content-length": "41", "date": "Tue, 03 Jun 2025 00:52:11 GMT"} })
26112025-06-03T00:52:11.078ZDEBGnexus (producer-registration-task): successfully registered with Nexus, pausing until time to renew lease lease_duration = 600s wait_period = 150s
26122025-06-03T00:52:11.112ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
26132025-06-03T00:52:11.197ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
26142025-06-03T00:52:11.230ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
26152025-06-03T00:52:11.231ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "9e3261da-4b5b-4984-b055-c53591c75717", "date": "Tue, 03 Jun 2025 00:52:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
26162025-06-03T00:52:11.231ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
26172025-06-03T00:52:11.232ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "3401400c-c65f-4dc4-9ee0-c162aaa33194", "date": "Tue, 03 Jun 2025 00:52:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
26182025-06-03T00:52:11.232ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
26192025-06-03T00:52:11.234ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "bd1e032a-c513-42c9-9cea-928a9dece521", "date": "Tue, 03 Jun 2025 00:52:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
26202025-06-03T00:52:11.234ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
26212025-06-03T00:52:11.236ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
26222025-06-03T00:52:11.236ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 391.221377ms iteration = 4 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
26232025-06-03T00:52:11.265ZDEBGnexus (BackgroundTasks): client request SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
26242025-06-03T00:52:11.266ZDEBGnexus (BackgroundTasks): client response SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "640d9185-b1d4-4660-b64e-29a8553e57a4", "content-length": "691", "date": "Tue, 03 Jun 2025 00:52:11 GMT"} })
26252025-06-03T00:52:11.266ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
26262025-06-03T00:52:11.266ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
26272025-06-03T00:52:11.266ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
26282025-06-03T00:52:11.267ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 758.913615ms iteration = 3 status = {}
26292025-06-03T00:52:11.358ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
26302025-06-03T00:52:11.359ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "7ace1d7b-645a-4625-b0b7-ec7609850568", "date": "Tue, 03 Jun 2025 00:52:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
26312025-06-03T00:52:11.359ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
26322025-06-03T00:52:11.361ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "80ea72c7-51d0-4572-9eb0-d5f09f20337d", "date": "Tue, 03 Jun 2025 00:52:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
26332025-06-03T00:52:11.361ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
26342025-06-03T00:52:11.362ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
26352025-06-03T00:52:11.362ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 502.363828ms iteration = 4 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
26362025-06-03T00:52:11.371ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
26372025-06-03T00:52:11.371ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
26382025-06-03T00:52:11.374ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: ec5e33f9-c697-46e2-a93e-a745336bf9dc, rack_id: d6831ba0-55f0-4191-8dc2-ea15e55ab0ae, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(9378a284-0ee9-48e2-80a0-3bcaadb1c4e2) } rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae switch_location = Switch0
26392025-06-03T00:52:11.374ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
26402025-06-03T00:52:11.374ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
26412025-06-03T00:52:11.385ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1832 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
26422025-06-03T00:52:11.385ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1859 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 9378a284-0ee9-48e2-80a0-3bcaadb1c4e2
26432025-06-03T00:52:11.385ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
26442025-06-03T00:52:11.385ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
26452025-06-03T00:52:11.385ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
26462025-06-03T00:52:11.385ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
26472025-06-03T00:52:11.385ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
26482025-06-03T00:52:11.385ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2006 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
26492025-06-03T00:52:11.385ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:408 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
26502025-06-03T00:52:11.385ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2042 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
26512025-06-03T00:52:11.385ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae
26522025-06-03T00:52:11.385ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
26532025-06-03T00:52:11.386ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
26542025-06-03T00:52:11.388ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae switch_location = Switch0
26552025-06-03T00:52:11.388ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
26562025-06-03T00:52:11.475ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "02b7104a-4177-44c8-b805-30b4330c8983", "date": "Tue, 03 Jun 2025 00:52:11 GMT"} })
26572025-06-03T00:52:11.475ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae switch = Switch0
26582025-06-03T00:52:11.475ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
26592025-06-03T00:52:11.476ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "3b764993-e246-4b72-822f-0030d8daddec", "content-length": "682", "date": "Tue, 03 Jun 2025 00:52:10 GMT"} })
26602025-06-03T00:52:11.476ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
26612025-06-03T00:52:11.476ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
26622025-06-03T00:52:11.479ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
26632025-06-03T00:52:11.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
26642025-06-03T00:52:11.483ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
26652025-06-03T00:52:11.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
26662025-06-03T00:52:11.485ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
26672025-06-03T00:52:11.486ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
26682025-06-03T00:52:11.500ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 991.581299ms iteration = 3 status = {}
26692025-06-03T00:52:20.508ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 8 reason = Timeout
26702025-06-03T00:52:20.508ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 8 reason = Timeout
26712025-06-03T00:52:20.508ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
26722025-06-03T00:52:20.508ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
26732025-06-03T00:52:20.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
26742025-06-03T00:52:20.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
26752025-06-03T00:52:20.542ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26762025-06-03T00:52:20.542ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
26772025-06-03T00:52:20.542ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
26782025-06-03T00:52:20.544ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
26792025-06-03T00:52:20.544ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
26802025-06-03T00:52:20.544ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
26812025-06-03T00:52:20.546ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
26822025-06-03T00:52:20.547ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
26832025-06-03T00:52:20.548ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26842025-06-03T00:52:20.551ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 4ec93699-353f-4b9e-ae06-25355fdb069e, lookup_type: ById(4ec93699-353f-4b9e-ae06-25355fdb069e) } result = Ok(())
26852025-06-03T00:52:20.551ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
26862025-06-03T00:52:20.551ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
26872025-06-03T00:52:20.560ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 4ec93699-353f-4b9e-ae06-25355fdb069e original_time_created = 2025-06-03 00:51:07.059326 UTC target_id = 4ec93699-353f-4b9e-ae06-25355fdb069e time_created = 2025-06-03 00:51:07.059326 UTC
26882025-06-03T00:52:20.560ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 52.001885ms iteration = 8 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"4ec93699-353f-4b9e-ae06-25355fdb069e","time_created":"2025-06-03T00:51:07.059326Z"}
26892025-06-03T00:52:20.576ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 68.018812ms iteration = 8 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
26902025-06-03T00:52:30.508ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 9 reason = Timeout
26912025-06-03T00:52:30.508ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
26922025-06-03T00:52:30.508ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 9 reason = Timeout
26932025-06-03T00:52:30.508ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
26942025-06-03T00:52:30.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
26952025-06-03T00:52:30.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
26962025-06-03T00:52:30.510ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26972025-06-03T00:52:30.510ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
26982025-06-03T00:52:30.510ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
26992025-06-03T00:52:30.512ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.535701ms iteration = 9 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
27002025-06-03T00:52:30.512ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
27012025-06-03T00:52:30.512ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
27022025-06-03T00:52:30.512ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
27032025-06-03T00:52:30.515ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
27042025-06-03T00:52:30.515ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
27052025-06-03T00:52:30.517ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27062025-06-03T00:52:30.519ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 4ec93699-353f-4b9e-ae06-25355fdb069e, lookup_type: ById(4ec93699-353f-4b9e-ae06-25355fdb069e) } result = Ok(())
27072025-06-03T00:52:30.519ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
27082025-06-03T00:52:30.519ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
27092025-06-03T00:52:30.530ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 4ec93699-353f-4b9e-ae06-25355fdb069e original_time_created = 2025-06-03 00:51:07.059326 UTC target_id = 4ec93699-353f-4b9e-ae06-25355fdb069e time_created = 2025-06-03 00:51:07.059326 UTC
27102025-06-03T00:52:30.530ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 21.996883ms iteration = 9 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"4ec93699-353f-4b9e-ae06-25355fdb069e","time_created":"2025-06-03T00:51:07.059326Z"}
27112025-06-03T00:52:35.675ZINFOnexus (dropshot): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:1025 local_addr = [fd00:1122:3344:101::a]:63576 remote_addr = [fd00:1122:3344:101::d]:61315
27122025-06-03T00:52:35.685ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 9759 local_addr = [fd00:1122:3344:101::a]:63576 method = GET remote_addr = [fd00:1122:3344:101::d]:61315 req_id = ea463f2b-0879-45a2-80b3-ba22567c5049 response_code = 200 uri = /a2861392-5fd2-462b-84e5-6f36c716263a
27132025-06-03T00:52:40.507ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 10 reason = Timeout
27142025-06-03T00:52:40.507ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 4 reason = Timeout
27152025-06-03T00:52:40.507ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
27162025-06-03T00:52:40.507ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 4 reason = Timeout
27172025-06-03T00:52:40.507ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
27182025-06-03T00:52:40.507ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
27192025-06-03T00:52:40.507ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
27202025-06-03T00:52:40.507ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 4 reason = Timeout
27212025-06-03T00:52:40.507ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
27222025-06-03T00:52:40.507ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 4 reason = Timeout
27232025-06-03T00:52:40.507ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 4 reason = Timeout
27242025-06-03T00:52:40.507ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
27252025-06-03T00:52:40.507ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
27262025-06-03T00:52:40.507ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 4 reason = Timeout
27272025-06-03T00:52:40.507ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 4 reason = Timeout
27282025-06-03T00:52:40.507ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 4 reason = Timeout
27292025-06-03T00:52:40.507ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
27302025-06-03T00:52:40.507ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
27312025-06-03T00:52:40.507ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
27322025-06-03T00:52:40.507ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
27332025-06-03T00:52:40.507ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 4 reason = Timeout
27342025-06-03T00:52:40.507ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 10 reason = Timeout
27352025-06-03T00:52:40.507ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
27362025-06-03T00:52:40.507ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
27372025-06-03T00:52:40.507ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 4 reason = Timeout
27382025-06-03T00:52:40.508ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
27392025-06-03T00:52:40.508ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 4 reason = Timeout
27402025-06-03T00:52:40.508ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 5 reason = Timeout
27412025-06-03T00:52:40.508ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
27422025-06-03T00:52:40.508ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
27432025-06-03T00:52:40.508ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 4 reason = Timeout
27442025-06-03T00:52:40.508ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
27452025-06-03T00:52:40.508ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 4 reason = Timeout
27462025-06-03T00:52:40.508ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
27472025-06-03T00:52:40.508ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 4 reason = Timeout
27482025-06-03T00:52:40.508ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
27492025-06-03T00:52:40.508ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 4 reason = Timeout
27502025-06-03T00:52:40.508ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 4 reason = Timeout
27512025-06-03T00:52:40.508ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
27522025-06-03T00:52:40.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
27532025-06-03T00:52:40.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
27542025-06-03T00:52:40.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
27552025-06-03T00:52:40.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
27562025-06-03T00:52:40.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
27572025-06-03T00:52:40.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
27582025-06-03T00:52:40.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
27592025-06-03T00:52:40.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
27602025-06-03T00:52:40.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
27612025-06-03T00:52:40.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
27622025-06-03T00:52:40.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
27632025-06-03T00:52:40.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
27642025-06-03T00:52:40.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
27652025-06-03T00:52:40.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
27662025-06-03T00:52:40.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
27672025-06-03T00:52:40.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
27682025-06-03T00:52:40.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
27692025-06-03T00:52:40.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
27702025-06-03T00:52:40.511ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27712025-06-03T00:52:40.512ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 4.162159ms iteration = 4 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
27722025-06-03T00:52:40.512ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-afe89ed7-19b7-479b-81eb-05c988ebc66d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-afe89ed7-19b7-479b-81eb-05c988ebc66d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 2487, tv_nsec: 734959161 } })
27732025-06-03T00:52:40.512ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-afe89ed7-19b7-479b-81eb-05c988ebc66d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-afe89ed7-19b7-479b-81eb-05c988ebc66d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 2487, tv_nsec: 735043414 } })
27742025-06-03T00:52:40.513ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1179
27752025-06-03T00:52:40.513ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1179
27762025-06-03T00:52:40.514ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
27772025-06-03T00:52:40.514ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
27782025-06-03T00:52:40.514ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
27792025-06-03T00:52:40.521ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27802025-06-03T00:52:40.523ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27812025-06-03T00:52:40.525ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
27822025-06-03T00:52:40.525ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
27832025-06-03T00:52:40.526ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
27842025-06-03T00:52:40.526ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
27852025-06-03T00:52:40.526ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
27862025-06-03T00:52:40.526ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
27872025-06-03T00:52:40.530ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27882025-06-03T00:52:40.532ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
27892025-06-03T00:52:40.532ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
27902025-06-03T00:52:40.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
27912025-06-03T00:52:40.537ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
27922025-06-03T00:52:40.538ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
27932025-06-03T00:52:40.538ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
27942025-06-03T00:52:40.538ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
27952025-06-03T00:52:40.571ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1187 zone_address = fd00:1122:3344:101::2
27962025-06-03T00:52:40.571ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
27972025-06-03T00:52:40.572ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "3fdb07be-699e-4154-862c-1c9d973b3958", "content-length": "26", "date": "Tue, 03 Jun 2025 00:52:40 GMT"} })
27982025-06-03T00:52:40.572ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1190 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
27992025-06-03T00:52:40.572ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
28002025-06-03T00:52:40.572ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
28012025-06-03T00:52:40.572ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
28022025-06-03T00:52:40.572ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
28032025-06-03T00:52:40.573ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28042025-06-03T00:52:40.573ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1224 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
28052025-06-03T00:52:40.573ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1187 zone_address = fd00:1122:3344:101::2
28062025-06-03T00:52:40.573ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
28072025-06-03T00:52:40.574ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "f33abb15-6046-4fb5-8eaf-23bc0fe0c9b8", "content-length": "26", "date": "Tue, 03 Jun 2025 00:52:40 GMT"} })
28082025-06-03T00:52:40.574ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1190 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
28092025-06-03T00:52:40.575ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1224 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
28102025-06-03T00:52:40.576ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
28112025-06-03T00:52:40.576ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
28122025-06-03T00:52:40.576ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
28132025-06-03T00:52:40.583ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28142025-06-03T00:52:40.584ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28152025-06-03T00:52:40.585ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
28162025-06-03T00:52:40.585ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
28172025-06-03T00:52:40.585ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
28182025-06-03T00:52:40.585ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
28192025-06-03T00:52:40.586ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
28202025-06-03T00:52:40.586ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
28212025-06-03T00:52:40.586ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
28222025-06-03T00:52:40.586ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
28232025-06-03T00:52:40.586ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
28242025-06-03T00:52:40.586ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
28252025-06-03T00:52:40.586ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
28262025-06-03T00:52:40.586ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
28272025-06-03T00:52:40.587ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
28282025-06-03T00:52:40.587ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
28292025-06-03T00:52:40.588ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 80.930424ms iteration = 4 status = {"errors":[],"requests_created_ok":[]}
28302025-06-03T00:52:40.588ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 81.141909ms iteration = 4 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
28312025-06-03T00:52:40.589ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 81.694016ms iteration = 4 status = {"errors":[],"finish_invoked_ok":[]}
28322025-06-03T00:52:40.592ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
28332025-06-03T00:52:40.592ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
28342025-06-03T00:52:40.601ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-afe89ed7-19b7-479b-81eb-05c988ebc66d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-afe89ed7-19b7-479b-81eb-05c988ebc66d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 2487, tv_nsec: 823393707 } })
28352025-06-03T00:52:40.601ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1179 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae
28362025-06-03T00:52:40.602ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 94.317639ms iteration = 4 status = {"errors":[],"garbage_collect_requested":[]}
28372025-06-03T00:52:40.604ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:547 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
28382025-06-03T00:52:40.604ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 96.046942ms iteration = 4 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
28392025-06-03T00:52:40.615ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28402025-06-03T00:52:40.615ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28412025-06-03T00:52:40.618ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 4ec93699-353f-4b9e-ae06-25355fdb069e, lookup_type: ById(4ec93699-353f-4b9e-ae06-25355fdb069e) } result = Ok(())
28422025-06-03T00:52:40.618ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
28432025-06-03T00:52:40.618ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
28442025-06-03T00:52:40.618ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
28452025-06-03T00:52:40.618ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
28462025-06-03T00:52:40.618ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
28472025-06-03T00:52:40.625ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
28482025-06-03T00:52:40.625ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
28492025-06-03T00:52:40.626ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
28502025-06-03T00:52:40.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
28512025-06-03T00:52:40.637ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
28522025-06-03T00:52:40.637ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
28532025-06-03T00:52:40.641ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 4ec93699-353f-4b9e-ae06-25355fdb069e original_time_created = 2025-06-03 00:51:07.059326 UTC target_id = 4ec93699-353f-4b9e-ae06-25355fdb069e time_created = 2025-06-03 00:51:07.059326 UTC
28542025-06-03T00:52:40.641ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 133.426933ms iteration = 10 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"4ec93699-353f-4b9e-ae06-25355fdb069e","time_created":"2025-06-03T00:51:07.059326Z"}
28552025-06-03T00:52:40.646ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 138.116639ms iteration = 10 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
28562025-06-03T00:52:40.647ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28572025-06-03T00:52:40.649ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
28582025-06-03T00:52:40.649ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
28592025-06-03T00:52:40.650ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
28602025-06-03T00:52:40.652ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 144.572099ms iteration = 4 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
28612025-06-03T00:52:40.658ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 150.283406ms iteration = 4 status = {}
28622025-06-03T00:52:40.660ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
28632025-06-03T00:52:40.660ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
28642025-06-03T00:52:40.662ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28652025-06-03T00:52:40.664ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
28662025-06-03T00:52:40.664ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
28672025-06-03T00:52:40.664ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
28682025-06-03T00:52:40.668ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:372
28692025-06-03T00:52:40.668ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 160.914236ms iteration = 4 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
28702025-06-03T00:52:40.671ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
28712025-06-03T00:52:40.671ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
28722025-06-03T00:52:40.671ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 164.126646ms iteration = 4 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
28732025-06-03T00:52:40.679ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 171.904268ms iteration = 4 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
28742025-06-03T00:52:40.692ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
28752025-06-03T00:52:40.692ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
28762025-06-03T00:52:40.702ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
28772025-06-03T00:52:40.702ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
28782025-06-03T00:52:40.704ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 196.42742ms iteration = 4 status = null
28792025-06-03T00:52:40.730ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1187 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae zone_address = fd00:1122:3344:101::2
28802025-06-03T00:52:40.730ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
28812025-06-03T00:52:40.730ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "7d3001eb-5255-46ac-a14d-b6f255119d91", "content-length": "26", "date": "Tue, 03 Jun 2025 00:52:40 GMT"} })
28822025-06-03T00:52:40.730ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1190 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
28832025-06-03T00:52:40.732ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1224 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae
28842025-06-03T00:52:40.754ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 246.393421ms iteration = 5 status = {"physical_disks_added":0}
28852025-06-03T00:52:40.754ZINFOnexus (BackgroundTasks): VPC route manager sled afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
28862025-06-03T00:52:40.754ZDEBGnexus (BackgroundTasks): client request SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
28872025-06-03T00:52:40.755ZDEBGnexus (BackgroundTasks): client response SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "320ca73e-95ff-4f4c-9873-a5a44c1cf1fc", "content-length": "372", "date": "Tue, 03 Jun 2025 00:52:40 GMT"} })
28882025-06-03T00:52:40.755ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
28892025-06-03T00:52:40.755ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
28902025-06-03T00:52:40.768ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
28912025-06-03T00:52:40.768ZDEBGnexus (BackgroundTasks): client request SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
28922025-06-03T00:52:40.768ZDEBGnexus (BackgroundTasks): client response SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "b238a9a4-53ae-4b43-a71d-57767daa2983", "date": "Tue, 03 Jun 2025 00:52:40 GMT"} })
28932025-06-03T00:52:40.768ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
28942025-06-03T00:52:40.769ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
28952025-06-03T00:52:40.771ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
28962025-06-03T00:52:40.771ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
28972025-06-03T00:52:40.773ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
28982025-06-03T00:52:40.774ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
28992025-06-03T00:52:40.795ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
29002025-06-03T00:52:40.796ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
29012025-06-03T00:52:40.797ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29022025-06-03T00:52:40.799ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
29032025-06-03T00:52:40.799ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
29042025-06-03T00:52:40.799ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
29052025-06-03T00:52:40.800ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29062025-06-03T00:52:40.802ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
29072025-06-03T00:52:40.802ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
29082025-06-03T00:52:40.802ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
29092025-06-03T00:52:40.834ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
29102025-06-03T00:52:40.835ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
29112025-06-03T00:52:40.836ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: ec5e33f9-c697-46e2-a93e-a745336bf9dc, rack_id: d6831ba0-55f0-4191-8dc2-ea15e55ab0ae, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(9378a284-0ee9-48e2-80a0-3bcaadb1c4e2) } rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae switch_location = Switch0
29122025-06-03T00:52:40.836ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
29132025-06-03T00:52:40.836ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
29142025-06-03T00:52:40.894ZINFOnexus (BackgroundTasks): VPC route manager sled afe89ed7-19b7-479b-81eb-05c988ebc66d push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
29152025-06-03T00:52:40.894ZDEBGnexus (BackgroundTasks): client request SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
29162025-06-03T00:52:40.909ZDEBGnexus (BackgroundTasks): client request SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
29172025-06-03T00:52:40.914ZDEBGnexus (BackgroundTasks): client response SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "0c073563-fc6c-4f65-9be2-a9dcbb9234ec", "date": "Tue, 03 Jun 2025 00:52:40 GMT"} })
29182025-06-03T00:52:40.916ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 408.321159ms iteration = 4 status = {}
29192025-06-03T00:52:40.916ZDEBGnexus (BackgroundTasks): client response SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "fe1c2ef5-9d79-4b94-998a-974f885b29ca", "content-length": "691", "date": "Tue, 03 Jun 2025 00:52:40 GMT"} })
29202025-06-03T00:52:40.916ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
29212025-06-03T00:52:40.916ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
29222025-06-03T00:52:40.916ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
29232025-06-03T00:52:40.917ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 409.263468ms iteration = 4 status = {}
29242025-06-03T00:52:40.955ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1832 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
29252025-06-03T00:52:40.955ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1859 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 9378a284-0ee9-48e2-80a0-3bcaadb1c4e2
29262025-06-03T00:52:40.956ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
29272025-06-03T00:52:40.956ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
29282025-06-03T00:52:40.956ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
29292025-06-03T00:52:40.956ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
29302025-06-03T00:52:40.956ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
29312025-06-03T00:52:40.956ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2006 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
29322025-06-03T00:52:40.956ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:408 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
29332025-06-03T00:52:40.956ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2042 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
29342025-06-03T00:52:40.956ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae
29352025-06-03T00:52:40.956ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
29362025-06-03T00:52:40.957ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
29372025-06-03T00:52:40.959ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae switch_location = Switch0
29382025-06-03T00:52:40.959ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
29392025-06-03T00:52:40.974ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
29402025-06-03T00:52:40.975ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
29412025-06-03T00:52:40.976ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
29422025-06-03T00:52:40.977ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
29432025-06-03T00:52:40.978ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29442025-06-03T00:52:40.981ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: afe89ed7-19b7-479b-81eb-05c988ebc66d, lookup_type: ById(afe89ed7-19b7-479b-81eb-05c988ebc66d) } result = Ok(())
29452025-06-03T00:52:40.981ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
29462025-06-03T00:52:40.981ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
29472025-06-03T00:52:40.982ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 475.057782ms iteration = 4 status = 0
29482025-06-03T00:52:41.116ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "cb2576b1-350e-4d23-a9fd-4c2c528f99e0", "date": "Tue, 03 Jun 2025 00:52:40 GMT"} })
29492025-06-03T00:52:41.116ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae switch = Switch0
29502025-06-03T00:52:41.116ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
29512025-06-03T00:52:41.116ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "953363f3-a2aa-4264-b650-7f9ed0b23ecc", "content-length": "682", "date": "Tue, 03 Jun 2025 00:52:40 GMT"} })
29522025-06-03T00:52:41.116ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
29532025-06-03T00:52:41.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
29542025-06-03T00:52:41.222ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
29552025-06-03T00:52:41.222ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
29562025-06-03T00:52:41.226ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
29572025-06-03T00:52:41.226ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
29582025-06-03T00:52:41.227ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
29592025-06-03T00:52:41.228ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
29602025-06-03T00:52:41.234ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 726.24469ms iteration = 4 status = {}
29612025-06-03T00:52:45.675ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 444 local_addr = [fd00:1122:3344:101::a]:63576 method = GET remote_addr = [fd00:1122:3344:101::d]:61315 req_id = 29e59f27-5196-4685-8624-4d1c6f7bd6aa response_code = 200 uri = /a2861392-5fd2-462b-84e5-6f36c716263a
29622025-06-03T00:52:50.507ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 11 reason = Timeout
29632025-06-03T00:52:50.507ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 11 reason = Timeout
29642025-06-03T00:52:50.507ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
29652025-06-03T00:52:50.507ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
29662025-06-03T00:52:50.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
29672025-06-03T00:52:50.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
29682025-06-03T00:52:50.509ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
29692025-06-03T00:52:50.509ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
29702025-06-03T00:52:50.510ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29712025-06-03T00:52:50.512ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
29722025-06-03T00:52:50.512ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
29732025-06-03T00:52:50.512ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
29742025-06-03T00:52:50.515ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
29752025-06-03T00:52:50.515ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
29762025-06-03T00:52:50.517ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29772025-06-03T00:52:50.519ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 4ec93699-353f-4b9e-ae06-25355fdb069e, lookup_type: ById(4ec93699-353f-4b9e-ae06-25355fdb069e) } result = Ok(())
29782025-06-03T00:52:50.519ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
29792025-06-03T00:52:50.519ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
29802025-06-03T00:52:50.529ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 4ec93699-353f-4b9e-ae06-25355fdb069e original_time_created = 2025-06-03 00:51:07.059326 UTC target_id = 4ec93699-353f-4b9e-ae06-25355fdb069e time_created = 2025-06-03 00:51:07.059326 UTC
29812025-06-03T00:52:50.529ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 22.227281ms iteration = 11 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"4ec93699-353f-4b9e-ae06-25355fdb069e","time_created":"2025-06-03T00:51:07.059326Z"}
29822025-06-03T00:52:50.543ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 35.924475ms iteration = 11 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
29832025-06-03T00:52:55.664ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:1025 local_addr = [fd00:1122:3344:101::a]:12221 remote_addr = [fd00:1122:3344:101::d]:64669
29842025-06-03T00:52:55.664ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:64669 req_id = ab6f009b-ffd5-4173-8080-e9501588cfab roles = RoleSet { roles: {} } uri = /metrics/collectors/292a19e1-4305-4e49-b11e-08ac74714bf3/producers?sort_by=id_ascending
29852025-06-03T00:52:55.664ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:64669 req_id = ab6f009b-ffd5-4173-8080-e9501588cfab resource = Database result = Ok(()) uri = /metrics/collectors/292a19e1-4305-4e49-b11e-08ac74714bf3/producers?sort_by=id_ascending
29862025-06-03T00:52:55.667ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 3347 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:64669 req_id = ab6f009b-ffd5-4173-8080-e9501588cfab response_code = 200 uri = /metrics/collectors/292a19e1-4305-4e49-b11e-08ac74714bf3/producers?sort_by=id_ascending
29872025-06-03T00:52:55.670ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:64669 req_id = 9bee5a5d-5553-453c-8a7a-d81858414690 roles = RoleSet { roles: {} } uri = /metrics/collectors/292a19e1-4305-4e49-b11e-08ac74714bf3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmZTEyZmZjMS1mMmFjLTRiZjMtOTY3Mi1mYzE4NWY4MmM1OWIifX0%3D
29882025-06-03T00:52:55.670ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:64669 req_id = 9bee5a5d-5553-453c-8a7a-d81858414690 resource = Database result = Ok(()) uri = /metrics/collectors/292a19e1-4305-4e49-b11e-08ac74714bf3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmZTEyZmZjMS1mMmFjLTRiZjMtOTY3Mi1mYzE4NWY4MmM1OWIifX0%3D
29892025-06-03T00:52:55.673ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 5236 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:64669 req_id = 9bee5a5d-5553-453c-8a7a-d81858414690 response_code = 200 uri = /metrics/collectors/292a19e1-4305-4e49-b11e-08ac74714bf3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmZTEyZmZjMS1mMmFjLTRiZjMtOTY3Mi1mYzE4NWY4MmM1OWIifX0%3D
29902025-06-03T00:52:55.675ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 236 local_addr = [fd00:1122:3344:101::a]:63576 method = GET remote_addr = [fd00:1122:3344:101::d]:61315 req_id = 6ec9b8da-bcfe-49b0-88d3-a87805357258 response_code = 200 uri = /a2861392-5fd2-462b-84e5-6f36c716263a
29912025-06-03T00:53:00.507ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 12 reason = Timeout
29922025-06-03T00:53:00.507ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 12 reason = Timeout
29932025-06-03T00:53:00.507ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
29942025-06-03T00:53:00.507ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
29952025-06-03T00:53:00.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
29962025-06-03T00:53:00.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
29972025-06-03T00:53:00.510ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
29982025-06-03T00:53:00.510ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
29992025-06-03T00:53:00.510ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30002025-06-03T00:53:00.511ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.892451ms iteration = 12 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
30012025-06-03T00:53:00.513ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
30022025-06-03T00:53:00.513ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
30032025-06-03T00:53:00.513ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
30042025-06-03T00:53:00.514ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
30052025-06-03T00:53:00.515ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
30062025-06-03T00:53:00.516ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30072025-06-03T00:53:00.519ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 4ec93699-353f-4b9e-ae06-25355fdb069e, lookup_type: ById(4ec93699-353f-4b9e-ae06-25355fdb069e) } result = Ok(())
30082025-06-03T00:53:00.519ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
30092025-06-03T00:53:00.519ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
30102025-06-03T00:53:00.630ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 4ec93699-353f-4b9e-ae06-25355fdb069e original_time_created = 2025-06-03 00:51:07.059326 UTC target_id = 4ec93699-353f-4b9e-ae06-25355fdb069e time_created = 2025-06-03 00:51:07.059326 UTC
30112025-06-03T00:53:00.630ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 122.432231ms iteration = 12 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"4ec93699-353f-4b9e-ae06-25355fdb069e","time_created":"2025-06-03T00:51:07.059326Z"}
30122025-06-03T00:53:05.675ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 308 local_addr = [fd00:1122:3344:101::a]:63576 method = GET remote_addr = [fd00:1122:3344:101::d]:61315 req_id = bbfaab49-9e5a-4df0-8949-2d9d309d7a18 response_code = 200 uri = /a2861392-5fd2-462b-84e5-6f36c716263a
30132025-06-03T00:53:10.506ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 3 reason = Timeout
30142025-06-03T00:53:10.506ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
30152025-06-03T00:53:10.506ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
30162025-06-03T00:53:10.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
30172025-06-03T00:53:10.506ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 13 reason = Timeout
30182025-06-03T00:53:10.506ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
30192025-06-03T00:53:10.506ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 5 reason = Timeout
30202025-06-03T00:53:10.506ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
30212025-06-03T00:53:10.506ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 5 reason = Timeout
30222025-06-03T00:53:10.506ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
30232025-06-03T00:53:10.506ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
30242025-06-03T00:53:10.506ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 3 reason = Timeout
30252025-06-03T00:53:10.506ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 5 reason = Timeout
30262025-06-03T00:53:10.506ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
30272025-06-03T00:53:10.507ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 5 reason = Timeout
30282025-06-03T00:53:10.507ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
30292025-06-03T00:53:10.507ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
30302025-06-03T00:53:10.507ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 5 reason = Timeout
30312025-06-03T00:53:10.507ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 5 reason = Timeout
30322025-06-03T00:53:10.507ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
30332025-06-03T00:53:10.507ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
30342025-06-03T00:53:10.507ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 3 reason = Timeout
30352025-06-03T00:53:10.507ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
30362025-06-03T00:53:10.507ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
30372025-06-03T00:53:10.507ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 5 reason = Timeout
30382025-06-03T00:53:10.507ZDEBGnexus (BackgroundTasks): activating background_task = alert_dispatcher iteration = 3 reason = Timeout
30392025-06-03T00:53:10.507ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
30402025-06-03T00:53:10.507ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
30412025-06-03T00:53:10.507ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 3 reason = Timeout
30422025-06-03T00:53:10.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
30432025-06-03T00:53:10.507ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 5 reason = Timeout
30442025-06-03T00:53:10.507ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
30452025-06-03T00:53:10.507ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
30462025-06-03T00:53:10.507ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 3 reason = Timeout
30472025-06-03T00:53:10.507ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
30482025-06-03T00:53:10.507ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 5 reason = Timeout
30492025-06-03T00:53:10.507ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
30502025-06-03T00:53:10.507ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 6 reason = Timeout
30512025-06-03T00:53:10.507ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
30522025-06-03T00:53:10.507ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 3 reason = Timeout
30532025-06-03T00:53:10.507ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:53
30542025-06-03T00:53:10.507ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
30552025-06-03T00:53:10.507ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 5 reason = Timeout
30562025-06-03T00:53:10.507ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
30572025-06-03T00:53:10.507ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 13 reason = Timeout
30582025-06-03T00:53:10.507ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
30592025-06-03T00:53:10.507ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 5 reason = Timeout
30602025-06-03T00:53:10.507ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
30612025-06-03T00:53:10.507ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 5 reason = Timeout
30622025-06-03T00:53:10.507ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
30632025-06-03T00:53:10.507ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 5 reason = Timeout
30642025-06-03T00:53:10.507ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
30652025-06-03T00:53:10.507ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 5 reason = Timeout
30662025-06-03T00:53:10.507ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 3 reason = Timeout
30672025-06-03T00:53:10.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
30682025-06-03T00:53:10.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
30692025-06-03T00:53:10.507ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
30702025-06-03T00:53:10.507ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 5 reason = Timeout
30712025-06-03T00:53:10.507ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
30722025-06-03T00:53:10.507ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 3 reason = Timeout
30732025-06-03T00:53:10.507ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 4 reason = Timeout
30742025-06-03T00:53:10.507ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:85 reason = blueprint disabled target_id = 4ec93699-353f-4b9e-ae06-25355fdb069e
30752025-06-03T00:53:10.507ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 5 reason = Timeout
30762025-06-03T00:53:10.507ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 10.431\u{b5}s iteration = 4 status = {"enabled":false,"target_id":"4ec93699-353f-4b9e-ae06-25355fdb069e"}
30772025-06-03T00:53:10.507ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 3 reason = Timeout
30782025-06-03T00:53:10.507ZDEBGnexus (BackgroundTasks): activating background_task = webhook_deliverator iteration = 3 reason = Timeout
30792025-06-03T00:53:10.507ZDEBGnexus (BackgroundTasks): roles background_task = webhook_deliverator roles = RoleSet { roles: {} }
30802025-06-03T00:53:10.507ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 3 reason = Timeout
30812025-06-03T00:53:10.507ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-06-03 00:43:10.507421063 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
30822025-06-03T00:53:10.507ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
30832025-06-03T00:53:10.507ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 5 reason = Timeout
30842025-06-03T00:53:10.507ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 5 reason = Timeout
30852025-06-03T00:53:10.507ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
30862025-06-03T00:53:10.507ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
30872025-06-03T00:53:10.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
30882025-06-03T00:53:10.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
30892025-06-03T00:53:10.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
30902025-06-03T00:53:10.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = alert_dispatcher resource = Database result = Ok(())
30912025-06-03T00:53:10.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
30922025-06-03T00:53:10.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
30932025-06-03T00:53:10.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
30942025-06-03T00:53:10.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
30952025-06-03T00:53:10.507ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
30962025-06-03T00:53:10.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
30972025-06-03T00:53:10.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
30982025-06-03T00:53:10.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
30992025-06-03T00:53:10.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
31002025-06-03T00:53:10.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
31012025-06-03T00:53:10.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
31022025-06-03T00:53:10.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
31032025-06-03T00:53:10.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
31042025-06-03T00:53:10.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
31052025-06-03T00:53:10.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
31062025-06-03T00:53:10.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
31072025-06-03T00:53:10.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
31082025-06-03T00:53:10.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
31092025-06-03T00:53:10.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = webhook_deliverator resource = Database result = Ok(())
31102025-06-03T00:53:10.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
31112025-06-03T00:53:10.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
31122025-06-03T00:53:10.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
31132025-06-03T00:53:10.508ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("241573bc-467c-4802-9f4b-f7c82da4e12e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("64fc0a69-d4b8-4f6c-9a07-77a4cfa94962.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("ceb74144-4b6b-4b3f-bc80-eac2d75f3bc9.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("ceb74144-4b6b-4b3f-bc80-eac2d75f3bc9.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:2::1))) }], valid_until: Instant { tv_sec: 2517, tv_nsec: 730953937 } })
31142025-06-03T00:53:10.508ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("9feb16c9-ff60-43e8-917b-965124f46e0d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("aa124775-31de-4648-981a-4af500268b66.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("aa124775-31de-4648-981a-4af500268b66.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::9))) }], valid_until: Instant { tv_sec: 2517, tv_nsec: 730957077 } })
31152025-06-03T00:53:10.508ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
31162025-06-03T00:53:10.508ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } dns_group = internal
31172025-06-03T00:53:10.508ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.29525ms iteration = 3 status = {"addresses":["[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:3::1]:5353","[fd00:1122:3344:2::1]:5353"]}
31182025-06-03T00:53:10.508ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.321611ms iteration = 3 status = {"addresses":["[fd00:1122:3344:101::8]:5353","[fd00:1122:3344:101::9]:5353"]}
31192025-06-03T00:53:10.509ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31202025-06-03T00:53:10.512ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
31212025-06-03T00:53:10.512ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
31222025-06-03T00:53:10.513ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
31232025-06-03T00:53:10.514ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-afe89ed7-19b7-479b-81eb-05c988ebc66d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-afe89ed7-19b7-479b-81eb-05c988ebc66d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 2517, tv_nsec: 737270693 } })
31242025-06-03T00:53:10.514ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1179
31252025-06-03T00:53:10.517ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31262025-06-03T00:53:10.517ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 10.209708ms iteration = 5 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
31272025-06-03T00:53:10.518ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31282025-06-03T00:53:10.518ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31292025-06-03T00:53:10.520ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
31302025-06-03T00:53:10.520ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
31312025-06-03T00:53:10.520ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
31322025-06-03T00:53:10.521ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
31332025-06-03T00:53:10.521ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
31342025-06-03T00:53:10.521ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
31352025-06-03T00:53:10.522ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
31362025-06-03T00:53:10.522ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
31372025-06-03T00:53:10.522ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
31382025-06-03T00:53:10.523ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 16.777271ms iteration = 3 status = {"errors":[],"found_port_ok":[]}
31392025-06-03T00:53:10.523ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31402025-06-03T00:53:10.526ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
31412025-06-03T00:53:10.526ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
31422025-06-03T00:53:10.526ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
31432025-06-03T00:53:10.529ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31442025-06-03T00:53:10.529ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31452025-06-03T00:53:10.532ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
31462025-06-03T00:53:10.532ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
31472025-06-03T00:53:10.532ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-afe89ed7-19b7-479b-81eb-05c988ebc66d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-afe89ed7-19b7-479b-81eb-05c988ebc66d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 2517, tv_nsec: 755503029 } })
31482025-06-03T00:53:10.532ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
31492025-06-03T00:53:10.532ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
31502025-06-03T00:53:10.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
31512025-06-03T00:53:10.533ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1179
31522025-06-03T00:53:10.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
31532025-06-03T00:53:10.535ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
31542025-06-03T00:53:10.535ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
31552025-06-03T00:53:10.535ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31562025-06-03T00:53:10.535ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
31572025-06-03T00:53:10.536ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
31582025-06-03T00:53:10.538ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-06-03 00:43:10.507421063 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
31592025-06-03T00:53:10.538ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 30.725384ms iteration = 3 status = {"expiration":"2025-06-03T00:43:10.507421063Z","pruned":[]}
31602025-06-03T00:53:10.539ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
31612025-06-03T00:53:10.539ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
31622025-06-03T00:53:10.539ZINFOnexus (BackgroundTasks): all webhook delivery tasks completed background_task = webhook_deliverator file = nexus/src/app/background/tasks/webhook_deliverator.rs:202 num_receivers = 0
31632025-06-03T00:53:10.539ZDEBGnexus (BackgroundTasks): activation complete background_task = webhook_deliverator elapsed = 31.794428ms iteration = 3 status = {"by_rx":{},"error":null}
31642025-06-03T00:53:10.539ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
31652025-06-03T00:53:10.541ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31662025-06-03T00:53:10.542ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
31672025-06-03T00:53:10.542ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
31682025-06-03T00:53:10.546ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
31692025-06-03T00:53:10.563ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
31702025-06-03T00:53:10.563ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
31712025-06-03T00:53:10.568ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
31722025-06-03T00:53:10.568ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
31732025-06-03T00:53:10.571ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31742025-06-03T00:53:10.573ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
31752025-06-03T00:53:10.574ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = alert_dispatcher resource = Database result = Ok(())
31762025-06-03T00:53:10.576ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
31772025-06-03T00:53:10.581ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
31782025-06-03T00:53:10.581ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
31792025-06-03T00:53:10.581ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
31802025-06-03T00:53:10.581ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
31812025-06-03T00:53:10.581ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 74.298897ms iteration = 13 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
31822025-06-03T00:53:10.581ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
31832025-06-03T00:53:10.582ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 75.529646ms iteration = 5 status = {"errors":[],"requests_created_ok":[]}
31842025-06-03T00:53:10.583ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
31852025-06-03T00:53:10.583ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
31862025-06-03T00:53:10.583ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
31872025-06-03T00:53:10.584ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
31882025-06-03T00:53:10.584ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31892025-06-03T00:53:10.584ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
31902025-06-03T00:53:10.584ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
31912025-06-03T00:53:10.588ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31922025-06-03T00:53:10.588ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 4ec93699-353f-4b9e-ae06-25355fdb069e, lookup_type: ById(4ec93699-353f-4b9e-ae06-25355fdb069e) } result = Ok(())
31932025-06-03T00:53:10.588ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
31942025-06-03T00:53:10.588ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31952025-06-03T00:53:10.588ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
31962025-06-03T00:53:10.592ZDEBGnexus (BackgroundTasks): activation complete background_task = alert_dispatcher elapsed = 85.048101ms iteration = 3 status = {"dispatched":[],"errors":[],"glob_version":"144.0.0","globs_reprocessed":{},"no_receivers":[]}
31972025-06-03T00:53:10.592ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
31982025-06-03T00:53:10.592ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
31992025-06-03T00:53:10.592ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
32002025-06-03T00:53:10.592ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
32012025-06-03T00:53:10.593ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
32022025-06-03T00:53:10.593ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
32032025-06-03T00:53:10.593ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
32042025-06-03T00:53:10.593ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
32052025-06-03T00:53:10.595ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32062025-06-03T00:53:10.596ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
32072025-06-03T00:53:10.596ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:167 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
32082025-06-03T00:53:10.596ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 89.489929ms iteration = 3 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
32092025-06-03T00:53:10.598ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
32102025-06-03T00:53:10.598ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
32112025-06-03T00:53:10.598ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
32122025-06-03T00:53:10.599ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
32132025-06-03T00:53:10.602ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32142025-06-03T00:53:10.602ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
32152025-06-03T00:53:10.602ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
32162025-06-03T00:53:10.602ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
32172025-06-03T00:53:10.605ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
32182025-06-03T00:53:10.605ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
32192025-06-03T00:53:10.605ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
32202025-06-03T00:53:10.609ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
32212025-06-03T00:53:10.609ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
32222025-06-03T00:53:10.615ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-afe89ed7-19b7-479b-81eb-05c988ebc66d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-afe89ed7-19b7-479b-81eb-05c988ebc66d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 2517, tv_nsec: 838825756 } })
32232025-06-03T00:53:10.616ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1179 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae
32242025-06-03T00:53:10.618ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 111.503342ms iteration = 5 status = {"errors":[],"finish_invoked_ok":[]}
32252025-06-03T00:53:10.619ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
32262025-06-03T00:53:10.619ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
32272025-06-03T00:53:10.619ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
32282025-06-03T00:53:10.619ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
32292025-06-03T00:53:10.619ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
32302025-06-03T00:53:10.620ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
32312025-06-03T00:53:10.624ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 4ec93699-353f-4b9e-ae06-25355fdb069e original_time_created = 2025-06-03 00:51:07.059326 UTC target_id = 4ec93699-353f-4b9e-ae06-25355fdb069e time_created = 2025-06-03 00:51:07.059326 UTC
32322025-06-03T00:53:10.624ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 117.331713ms iteration = 13 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"4ec93699-353f-4b9e-ae06-25355fdb069e","time_created":"2025-06-03T00:51:07.059326Z"}
32332025-06-03T00:53:10.624ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
32342025-06-03T00:53:10.625ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
32352025-06-03T00:53:10.635ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32362025-06-03T00:53:10.635ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32372025-06-03T00:53:10.637ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
32382025-06-03T00:53:10.637ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:137 found = 0 not_ready_to_be_deleted = 0
32392025-06-03T00:53:10.637ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 130.367418ms iteration = 3 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
32402025-06-03T00:53:10.638ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
32412025-06-03T00:53:10.638ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
32422025-06-03T00:53:10.638ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
32432025-06-03T00:53:10.640ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
32442025-06-03T00:53:10.640ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
32452025-06-03T00:53:10.640ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
32462025-06-03T00:53:10.640ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
32472025-06-03T00:53:10.640ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
32482025-06-03T00:53:10.642ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 135.087355ms iteration = 5 status = {"errors":[],"garbage_collect_requested":[]}
32492025-06-03T00:53:10.643ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
32502025-06-03T00:53:10.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
32512025-06-03T00:53:10.645ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1187 zone_address = fd00:1122:3344:101::2
32522025-06-03T00:53:10.645ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
32532025-06-03T00:53:10.645ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "27e1009a-137a-499b-a9a5-304ba6c55dd8", "content-length": "26", "date": "Tue, 03 Jun 2025 00:53:10 GMT"} })
32542025-06-03T00:53:10.645ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1190 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
32552025-06-03T00:53:10.646ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 139.118609ms iteration = 5 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
32562025-06-03T00:53:10.647ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1224 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
32572025-06-03T00:53:10.648ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
32582025-06-03T00:53:10.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
32592025-06-03T00:53:10.649ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:547 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
32602025-06-03T00:53:10.649ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 142.875236ms iteration = 5 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
32612025-06-03T00:53:10.658ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32622025-06-03T00:53:10.661ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
32632025-06-03T00:53:10.661ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
32642025-06-03T00:53:10.661ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
32652025-06-03T00:53:10.667ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1187 zone_address = fd00:1122:3344:101::2
32662025-06-03T00:53:10.667ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
32672025-06-03T00:53:10.668ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "601ad040-41e1-4240-8b78-7647ee24b41d", "content-length": "26", "date": "Tue, 03 Jun 2025 00:53:10 GMT"} })
32682025-06-03T00:53:10.668ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1190 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
32692025-06-03T00:53:10.671ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
32702025-06-03T00:53:10.671ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1224 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
32712025-06-03T00:53:10.678ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
32722025-06-03T00:53:10.678ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
32732025-06-03T00:53:10.688ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 181.435903ms iteration = 5 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
32742025-06-03T00:53:10.695ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
32752025-06-03T00:53:10.695ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
32762025-06-03T00:53:10.696ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
32772025-06-03T00:53:10.696ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 3ef110b4-9400-4c37-8d7f-5552c8a94db0 dns_zone_name = oxide.test found_so_far = 0 version = 2
32782025-06-03T00:53:10.696ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
32792025-06-03T00:53:10.696ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
32802025-06-03T00:53:10.698ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
32812025-06-03T00:53:10.699ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
32822025-06-03T00:53:10.707ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
32832025-06-03T00:53:10.707ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 9e79f9b1-a4de-4f9d-a901-16cc4f79e15d dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
32842025-06-03T00:53:10.707ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
32852025-06-03T00:53:10.707ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
32862025-06-03T00:53:10.708ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32872025-06-03T00:53:10.709ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32882025-06-03T00:53:10.710ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
32892025-06-03T00:53:10.711ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
32902025-06-03T00:53:10.711ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
32912025-06-03T00:53:10.711ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
32922025-06-03T00:53:10.711ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
32932025-06-03T00:53:10.711ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
32942025-06-03T00:53:10.713ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
32952025-06-03T00:53:10.713ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
32962025-06-03T00:53:10.713ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 206.979557ms iteration = 5 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
32972025-06-03T00:53:10.713ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
32982025-06-03T00:53:10.714ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
32992025-06-03T00:53:10.765ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
33002025-06-03T00:53:10.765ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "9d3062d5-bf04-43c8-96de-767c1677d2da", "date": "Tue, 03 Jun 2025 00:53:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
33012025-06-03T00:53:10.765ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
33022025-06-03T00:53:10.767ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "2875a426-d058-48b7-ac1b-33cebd14549a", "date": "Tue, 03 Jun 2025 00:53:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
33032025-06-03T00:53:10.767ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
33042025-06-03T00:53:10.768ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
33052025-06-03T00:53:10.768ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 261.549432ms iteration = 5 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
33062025-06-03T00:53:10.784ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 277.142526ms iteration = 6 status = {"physical_disks_added":0}
33072025-06-03T00:53:10.816ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 309.532951ms iteration = 5 status = {}
33082025-06-03T00:53:10.819ZINFOnexus (BackgroundTasks): VPC route manager sled afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
33092025-06-03T00:53:10.819ZDEBGnexus (BackgroundTasks): client request SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
33102025-06-03T00:53:10.819ZDEBGnexus (BackgroundTasks): client response SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "06843f16-55d1-451a-945a-be3b6108bbfa", "content-length": "372", "date": "Tue, 03 Jun 2025 00:53:10 GMT"} })
33112025-06-03T00:53:10.819ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
33122025-06-03T00:53:10.820ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
33132025-06-03T00:53:10.823ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1187 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae zone_address = fd00:1122:3344:101::2
33142025-06-03T00:53:10.823ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
33152025-06-03T00:53:10.825ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "c5d8e435-8c16-4446-a180-8609bbae032d", "content-length": "26", "date": "Tue, 03 Jun 2025 00:53:10 GMT"} })
33162025-06-03T00:53:10.825ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1190 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
33172025-06-03T00:53:10.827ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1224 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae
33182025-06-03T00:53:10.829ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33192025-06-03T00:53:10.830ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33202025-06-03T00:53:10.830ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
33212025-06-03T00:53:10.831ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
33222025-06-03T00:53:10.833ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
33232025-06-03T00:53:10.833ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
33242025-06-03T00:53:10.833ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
33252025-06-03T00:53:10.833ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
33262025-06-03T00:53:10.833ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
33272025-06-03T00:53:10.834ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
33282025-06-03T00:53:10.836ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33292025-06-03T00:53:10.838ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
33302025-06-03T00:53:10.840ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
33312025-06-03T00:53:10.840ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
33322025-06-03T00:53:10.840ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
33332025-06-03T00:53:10.844ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 3ef110b4-9400-4c37-8d7f-5552c8a94db0 dns_zone_name = oxide.test found_so_far = 4 version = 2
33342025-06-03T00:53:10.844ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
33352025-06-03T00:53:10.844ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2025-06-03 00:51:38.567192 UTC dns_group = external generation = 2
33362025-06-03T00:53:10.844ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 336.889461ms iteration = 3 status = {"generation":2}
33372025-06-03T00:53:10.852ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
33382025-06-03T00:53:10.852ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
33392025-06-03T00:53:10.854ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 347.205821ms iteration = 5 status = null
33402025-06-03T00:53:10.878ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:372
33412025-06-03T00:53:10.878ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 371.743824ms iteration = 5 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
33422025-06-03T00:53:10.884ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 376.906863ms iteration = 5 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
33432025-06-03T00:53:10.893ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
33442025-06-03T00:53:10.893ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33452025-06-03T00:53:10.894ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "de1c60e1-1a77-4f0c-a33e-cdb5a45a38ac", "date": "Tue, 03 Jun 2025 00:53:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
33462025-06-03T00:53:10.894ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
33472025-06-03T00:53:10.896ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "583f721e-46b4-4915-a96e-ab76f82ff851", "date": "Tue, 03 Jun 2025 00:53:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
33482025-06-03T00:53:10.896ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
33492025-06-03T00:53:10.896ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: afe89ed7-19b7-479b-81eb-05c988ebc66d, lookup_type: ById(afe89ed7-19b7-479b-81eb-05c988ebc66d) } result = Ok(())
33502025-06-03T00:53:10.896ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
33512025-06-03T00:53:10.896ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
33522025-06-03T00:53:10.897ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "91cc3fd2-81e2-43ed-a609-46b0b0b1c797", "date": "Tue, 03 Jun 2025 00:53:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
33532025-06-03T00:53:10.897ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
33542025-06-03T00:53:10.898ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 391.419604ms iteration = 5 status = 0
33552025-06-03T00:53:10.898ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-06-03T00:51:07.585932Z, time_modified: 2025-06-03T00:51:07.585932Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: ea0ef520-6bf3-4fbd-9ff0-f3698b48b749, db_silo: Silo { identity: SiloIdentity { id: ea0ef520-6bf3-4fbd-9ff0-f3698b48b749, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-06-03T00:51:38.523369Z, time_modified: 2025-06-03T00:51:38.523369Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "bcda8a9f16e63c3cec49f9759478b9f335a661639158b0a9026f69b846d4e127" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: ea0ef520-6bf3-4fbd-9ff0-f3698b48b749, db_silo: Silo { identity: SiloIdentity { id: ea0ef520-6bf3-4fbd-9ff0-f3698b48b749, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-06-03T00:51:38.523369Z, time_modified: 2025-06-03T00:51:38.523369Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "bcda8a9f16e63c3cec49f9759478b9f335a661639158b0a9026f69b846d4e127" }] }) }
33562025-06-03T00:53:10.899ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 391.985632ms iteration = 3 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"ea0ef520-6bf3-4fbd-9ff0-f3698b48b749","tls_certs":["bcda8a9f16e63c3cec49f9759478b9f335a661639158b0a9026f69b846d4e127"]}},"default_endpoint":{"silo_id":"ea0ef520-6bf3-4fbd-9ff0-f3698b48b749","tls_certs":["bcda8a9f16e63c3cec49f9759478b9f335a661639158b0a9026f69b846d4e127"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
33572025-06-03T00:53:10.899ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
33582025-06-03T00:53:10.899ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 391.742344ms iteration = 5 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
33592025-06-03T00:53:10.900ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
33602025-06-03T00:53:10.900ZDEBGnexus (BackgroundTasks): client request SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
33612025-06-03T00:53:10.900ZDEBGnexus (BackgroundTasks): client response SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "bb584460-1110-4ec6-9068-010c57c0afe8", "date": "Tue, 03 Jun 2025 00:53:10 GMT"} })
33622025-06-03T00:53:10.900ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
33632025-06-03T00:53:10.901ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
33642025-06-03T00:53:10.903ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
33652025-06-03T00:53:10.904ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
33662025-06-03T00:53:10.908ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
33672025-06-03T00:53:10.908ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
33682025-06-03T00:53:10.912ZINFOnexus (BackgroundTasks): VPC route manager sled afe89ed7-19b7-479b-81eb-05c988ebc66d push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
33692025-06-03T00:53:10.912ZDEBGnexus (BackgroundTasks): client request SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
33702025-06-03T00:53:10.912ZDEBGnexus (BackgroundTasks): client response SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "4d872333-aa19-44a3-9964-94093ea26a8c", "date": "Tue, 03 Jun 2025 00:53:10 GMT"} })
33712025-06-03T00:53:10.914ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 407.7082ms iteration = 5 status = {}
33722025-06-03T00:53:10.949ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 9e79f9b1-a4de-4f9d-a901-16cc4f79e15d dns_zone_name = control-plane.oxide.internal found_so_far = 59 version = 1
33732025-06-03T00:53:10.949ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
33742025-06-03T00:53:10.949ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2025-06-03 00:51:38.152773 UTC dns_group = internal generation = 1
33752025-06-03T00:53:10.949ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 442.994106ms iteration = 3 status = {"generation":1}
33762025-06-03T00:53:10.975ZDEBGnexus (BackgroundTasks): client request SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
33772025-06-03T00:53:10.985ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
33782025-06-03T00:53:10.985ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
33792025-06-03T00:53:10.987ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: ec5e33f9-c697-46e2-a93e-a745336bf9dc, rack_id: d6831ba0-55f0-4191-8dc2-ea15e55ab0ae, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(9378a284-0ee9-48e2-80a0-3bcaadb1c4e2) } rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae switch_location = Switch0
33802025-06-03T00:53:10.987ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
33812025-06-03T00:53:10.987ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
33822025-06-03T00:53:10.989ZINFOnexus (BackgroundTasks): instance reincarnation completed successfully background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:113 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0
33832025-06-03T00:53:10.989ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 482.140241ms iteration = 3 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
33842025-06-03T00:53:10.999ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1832 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
33852025-06-03T00:53:10.999ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1859 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 9378a284-0ee9-48e2-80a0-3bcaadb1c4e2
33862025-06-03T00:53:10.999ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
33872025-06-03T00:53:10.999ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
33882025-06-03T00:53:10.999ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
33892025-06-03T00:53:10.999ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
33902025-06-03T00:53:10.999ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
33912025-06-03T00:53:10.999ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2006 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
33922025-06-03T00:53:10.999ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:408 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
33932025-06-03T00:53:10.999ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2042 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
33942025-06-03T00:53:10.999ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae
33952025-06-03T00:53:10.999ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
33962025-06-03T00:53:11.000ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
33972025-06-03T00:53:11.001ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae switch_location = Switch0
33982025-06-03T00:53:11.001ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
33992025-06-03T00:53:11.063ZDEBGnexus (BackgroundTasks): client response SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "53ea464d-eaf8-4346-932a-93fba7b84204", "content-length": "691", "date": "Tue, 03 Jun 2025 00:53:11 GMT"} })
34002025-06-03T00:53:11.063ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
34012025-06-03T00:53:11.063ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
34022025-06-03T00:53:11.063ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
34032025-06-03T00:53:11.065ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 558.006257ms iteration = 5 status = {}
34042025-06-03T00:53:11.152ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "e3054975-da7b-4474-b096-5952ad910185", "date": "Tue, 03 Jun 2025 00:53:11 GMT"} })
34052025-06-03T00:53:11.152ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae switch = Switch0
34062025-06-03T00:53:11.152ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
34072025-06-03T00:53:11.155ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "481690cc-6372-4667-b35d-655ef86f0bbf", "content-length": "682", "date": "Tue, 03 Jun 2025 00:53:11 GMT"} })
34082025-06-03T00:53:11.155ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
34092025-06-03T00:53:11.156ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
34102025-06-03T00:53:11.159ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
34112025-06-03T00:53:11.160ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
34122025-06-03T00:53:11.163ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
34132025-06-03T00:53:11.164ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
34142025-06-03T00:53:11.165ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
34152025-06-03T00:53:11.165ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
34162025-06-03T00:53:11.172ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 665.511275ms iteration = 5 status = {}
34172025-06-03T00:53:15.674ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 250 local_addr = [fd00:1122:3344:101::a]:63576 method = GET remote_addr = [fd00:1122:3344:101::d]:61315 req_id = e72d16f5-783b-4468-97ed-4f64b7ee8ea2 response_code = 200 uri = /a2861392-5fd2-462b-84e5-6f36c716263a
34182025-06-03T00:53:20.506ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 14 reason = Timeout
34192025-06-03T00:53:20.506ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 14 reason = Timeout
34202025-06-03T00:53:20.506ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
34212025-06-03T00:53:20.506ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
34222025-06-03T00:53:20.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
34232025-06-03T00:53:20.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
34242025-06-03T00:53:20.509ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34252025-06-03T00:53:20.509ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
34262025-06-03T00:53:20.510ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
34272025-06-03T00:53:20.511ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
34282025-06-03T00:53:20.511ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
34292025-06-03T00:53:20.511ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
34302025-06-03T00:53:20.514ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
34312025-06-03T00:53:20.514ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
34322025-06-03T00:53:20.516ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34332025-06-03T00:53:20.519ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 4ec93699-353f-4b9e-ae06-25355fdb069e, lookup_type: ById(4ec93699-353f-4b9e-ae06-25355fdb069e) } result = Ok(())
34342025-06-03T00:53:20.519ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
34352025-06-03T00:53:20.519ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
34362025-06-03T00:53:20.531ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 4ec93699-353f-4b9e-ae06-25355fdb069e original_time_created = 2025-06-03 00:51:07.059326 UTC target_id = 4ec93699-353f-4b9e-ae06-25355fdb069e time_created = 2025-06-03 00:51:07.059326 UTC
34372025-06-03T00:53:20.531ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 24.609215ms iteration = 14 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"4ec93699-353f-4b9e-ae06-25355fdb069e","time_created":"2025-06-03T00:51:07.059326Z"}
34382025-06-03T00:53:20.549ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 42.660865ms iteration = 14 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
34392025-06-03T00:53:25.674ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 293 local_addr = [fd00:1122:3344:101::a]:63576 method = GET remote_addr = [fd00:1122:3344:101::d]:61315 req_id = b657a349-5baf-4a63-b16a-b2735c0fd0a8 response_code = 200 uri = /a2861392-5fd2-462b-84e5-6f36c716263a
34402025-06-03T00:53:30.506ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 15 reason = Timeout
34412025-06-03T00:53:30.506ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
34422025-06-03T00:53:30.506ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 15 reason = Timeout
34432025-06-03T00:53:30.506ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
34442025-06-03T00:53:30.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
34452025-06-03T00:53:30.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
34462025-06-03T00:53:30.508ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
34472025-06-03T00:53:30.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
34482025-06-03T00:53:30.509ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34492025-06-03T00:53:30.511ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
34502025-06-03T00:53:30.511ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
34512025-06-03T00:53:30.511ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
34522025-06-03T00:53:30.512ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
34532025-06-03T00:53:30.513ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
34542025-06-03T00:53:30.514ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34552025-06-03T00:53:30.517ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 4ec93699-353f-4b9e-ae06-25355fdb069e, lookup_type: ById(4ec93699-353f-4b9e-ae06-25355fdb069e) } result = Ok(())
34562025-06-03T00:53:30.517ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
34572025-06-03T00:53:30.517ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
34582025-06-03T00:53:30.541ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 4ec93699-353f-4b9e-ae06-25355fdb069e original_time_created = 2025-06-03 00:51:07.059326 UTC target_id = 4ec93699-353f-4b9e-ae06-25355fdb069e time_created = 2025-06-03 00:51:07.059326 UTC
34592025-06-03T00:53:30.541ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 35.156851ms iteration = 15 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"4ec93699-353f-4b9e-ae06-25355fdb069e","time_created":"2025-06-03T00:51:07.059326Z"}
34602025-06-03T00:53:30.542ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 36.341398ms iteration = 15 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
34612025-06-03T00:53:35.674ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 352 local_addr = [fd00:1122:3344:101::a]:63576 method = GET remote_addr = [fd00:1122:3344:101::d]:61315 req_id = 693cdc1d-3b41-4394-b9cf-cd3072978446 response_code = 200 uri = /a2861392-5fd2-462b-84e5-6f36c716263a
34622025-06-03T00:53:40.506ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 16 reason = Timeout
34632025-06-03T00:53:40.506ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 7 reason = Timeout
34642025-06-03T00:53:40.506ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
34652025-06-03T00:53:40.506ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
34662025-06-03T00:53:40.506ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 6 reason = Timeout
34672025-06-03T00:53:40.506ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 6 reason = Timeout
34682025-06-03T00:53:40.506ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
34692025-06-03T00:53:40.506ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
34702025-06-03T00:53:40.506ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 6 reason = Timeout
34712025-06-03T00:53:40.506ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
34722025-06-03T00:53:40.506ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 6 reason = Timeout
34732025-06-03T00:53:40.506ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
34742025-06-03T00:53:40.506ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 6 reason = Timeout
34752025-06-03T00:53:40.506ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
34762025-06-03T00:53:40.506ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 6 reason = Timeout
34772025-06-03T00:53:40.506ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
34782025-06-03T00:53:40.506ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
34792025-06-03T00:53:40.506ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 6 reason = Timeout
34802025-06-03T00:53:40.506ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
34812025-06-03T00:53:40.506ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 6 reason = Timeout
34822025-06-03T00:53:40.506ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
34832025-06-03T00:53:40.506ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 6 reason = Timeout
34842025-06-03T00:53:40.506ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
34852025-06-03T00:53:40.506ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 6 reason = Timeout
34862025-06-03T00:53:40.506ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
34872025-06-03T00:53:40.506ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 6 reason = Timeout
34882025-06-03T00:53:40.506ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
34892025-06-03T00:53:40.506ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 6 reason = Timeout
34902025-06-03T00:53:40.506ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
34912025-06-03T00:53:40.506ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
34922025-06-03T00:53:40.506ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 6 reason = Timeout
34932025-06-03T00:53:40.506ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
34942025-06-03T00:53:40.506ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 6 reason = Timeout
34952025-06-03T00:53:40.506ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 6 reason = Timeout
34962025-06-03T00:53:40.506ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
34972025-06-03T00:53:40.507ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 16 reason = Timeout
34982025-06-03T00:53:40.507ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 6 reason = Timeout
34992025-06-03T00:53:40.507ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
35002025-06-03T00:53:40.507ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
35012025-06-03T00:53:40.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
35022025-06-03T00:53:40.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
35032025-06-03T00:53:40.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
35042025-06-03T00:53:40.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
35052025-06-03T00:53:40.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
35062025-06-03T00:53:40.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
35072025-06-03T00:53:40.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
35082025-06-03T00:53:40.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
35092025-06-03T00:53:40.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
35102025-06-03T00:53:40.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
35112025-06-03T00:53:40.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
35122025-06-03T00:53:40.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
35132025-06-03T00:53:40.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
35142025-06-03T00:53:40.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
35152025-06-03T00:53:40.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
35162025-06-03T00:53:40.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
35172025-06-03T00:53:40.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
35182025-06-03T00:53:40.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
35192025-06-03T00:53:40.510ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
35202025-06-03T00:53:40.510ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
35212025-06-03T00:53:40.510ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35222025-06-03T00:53:40.511ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35232025-06-03T00:53:40.512ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
35242025-06-03T00:53:40.512ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
35252025-06-03T00:53:40.512ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
35262025-06-03T00:53:40.512ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 6.321197ms iteration = 6 status = {"errors":[],"garbage_collect_requested":[]}
35272025-06-03T00:53:40.513ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35282025-06-03T00:53:40.513ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 7.204144ms iteration = 6 status = {"errors":[],"finish_invoked_ok":[]}
35292025-06-03T00:53:40.515ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
35302025-06-03T00:53:40.516ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
35312025-06-03T00:53:40.516ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
35322025-06-03T00:53:40.516ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
35332025-06-03T00:53:40.516ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
35342025-06-03T00:53:40.516ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 9.512715ms iteration = 6 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
35352025-06-03T00:53:40.516ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
35362025-06-03T00:53:40.521ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-afe89ed7-19b7-479b-81eb-05c988ebc66d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-afe89ed7-19b7-479b-81eb-05c988ebc66d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 2547, tv_nsec: 744358773 } })
35372025-06-03T00:53:40.521ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1179
35382025-06-03T00:53:40.523ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35392025-06-03T00:53:40.525ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
35402025-06-03T00:53:40.525ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
35412025-06-03T00:53:40.525ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
35422025-06-03T00:53:40.527ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35432025-06-03T00:53:40.528ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
35442025-06-03T00:53:40.528ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
35452025-06-03T00:53:40.528ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
35462025-06-03T00:53:40.528ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
35472025-06-03T00:53:40.529ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35482025-06-03T00:53:40.530ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
35492025-06-03T00:53:40.530ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
35502025-06-03T00:53:40.530ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
35512025-06-03T00:53:40.531ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
35522025-06-03T00:53:40.531ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
35532025-06-03T00:53:40.531ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
35542025-06-03T00:53:40.532ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
35552025-06-03T00:53:40.532ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
35562025-06-03T00:53:40.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
35572025-06-03T00:53:40.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
35582025-06-03T00:53:40.534ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
35592025-06-03T00:53:40.535ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
35602025-06-03T00:53:40.537ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-afe89ed7-19b7-479b-81eb-05c988ebc66d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-afe89ed7-19b7-479b-81eb-05c988ebc66d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 2547, tv_nsec: 760407142 } })
35612025-06-03T00:53:40.537ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1179 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae
35622025-06-03T00:53:40.539ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35632025-06-03T00:53:40.541ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
35642025-06-03T00:53:40.541ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
35652025-06-03T00:53:40.541ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
35662025-06-03T00:53:40.571ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
35672025-06-03T00:53:40.571ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
35682025-06-03T00:53:40.581ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
35692025-06-03T00:53:40.581ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
35702025-06-03T00:53:40.584ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
35712025-06-03T00:53:40.585ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 78.444606ms iteration = 6 status = {"errors":[],"requests_created_ok":[]}
35722025-06-03T00:53:40.585ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
35732025-06-03T00:53:40.585ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 78.984142ms iteration = 16 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
35742025-06-03T00:53:40.585ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35752025-06-03T00:53:40.587ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 80.402296ms iteration = 6 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
35762025-06-03T00:53:40.588ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
35772025-06-03T00:53:40.588ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
35782025-06-03T00:53:40.588ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
35792025-06-03T00:53:40.589ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
35802025-06-03T00:53:40.589ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
35812025-06-03T00:53:40.589ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
35822025-06-03T00:53:40.589ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
35832025-06-03T00:53:40.589ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1187 zone_address = fd00:1122:3344:101::2
35842025-06-03T00:53:40.589ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
35852025-06-03T00:53:40.590ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "0b9aa517-d9dc-44c6-a84b-e7366dd5a475", "content-length": "26", "date": "Tue, 03 Jun 2025 00:53:40 GMT"} })
35862025-06-03T00:53:40.590ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1190 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
35872025-06-03T00:53:40.591ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1224 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
35882025-06-03T00:53:40.591ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35892025-06-03T00:53:40.593ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
35902025-06-03T00:53:40.593ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
35912025-06-03T00:53:40.594ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35922025-06-03T00:53:40.594ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 4ec93699-353f-4b9e-ae06-25355fdb069e, lookup_type: ById(4ec93699-353f-4b9e-ae06-25355fdb069e) } result = Ok(())
35932025-06-03T00:53:40.594ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
35942025-06-03T00:53:40.595ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
35952025-06-03T00:53:40.597ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
35962025-06-03T00:53:40.597ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
35972025-06-03T00:53:40.597ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
35982025-06-03T00:53:40.598ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 91.288645ms iteration = 6 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
35992025-06-03T00:53:40.615ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 4ec93699-353f-4b9e-ae06-25355fdb069e original_time_created = 2025-06-03 00:51:07.059326 UTC target_id = 4ec93699-353f-4b9e-ae06-25355fdb069e time_created = 2025-06-03 00:51:07.059326 UTC
36002025-06-03T00:53:40.615ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 108.272202ms iteration = 16 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"4ec93699-353f-4b9e-ae06-25355fdb069e","time_created":"2025-06-03T00:51:07.059326Z"}
36012025-06-03T00:53:40.618ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1187 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae zone_address = fd00:1122:3344:101::2
36022025-06-03T00:53:40.618ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
36032025-06-03T00:53:40.618ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "16903610-a9e1-4f67-91fa-61b37fc558df", "content-length": "26", "date": "Tue, 03 Jun 2025 00:53:40 GMT"} })
36042025-06-03T00:53:40.618ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1190 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
36052025-06-03T00:53:40.619ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
36062025-06-03T00:53:40.619ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
36072025-06-03T00:53:40.620ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1224 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae
36082025-06-03T00:53:40.621ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 114.743413ms iteration = 6 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
36092025-06-03T00:53:40.628ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36102025-06-03T00:53:40.628ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-afe89ed7-19b7-479b-81eb-05c988ebc66d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-afe89ed7-19b7-479b-81eb-05c988ebc66d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 2547, tv_nsec: 851700136 } })
36112025-06-03T00:53:40.628ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1179
36122025-06-03T00:53:40.629ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
36132025-06-03T00:53:40.629ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
36142025-06-03T00:53:40.630ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
36152025-06-03T00:53:40.631ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:372
36162025-06-03T00:53:40.631ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 124.985331ms iteration = 6 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
36172025-06-03T00:53:40.641ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
36182025-06-03T00:53:40.641ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
36192025-06-03T00:53:40.641ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 134.579869ms iteration = 6 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
36202025-06-03T00:53:40.643ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:547 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
36212025-06-03T00:53:40.643ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 136.985444ms iteration = 6 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
36222025-06-03T00:53:40.651ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
36232025-06-03T00:53:40.651ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
36242025-06-03T00:53:40.653ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 146.651703ms iteration = 6 status = null
36252025-06-03T00:53:40.663ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:1025 local_addr = [fd00:1122:3344:101::a]:12221 remote_addr = [fd00:1122:3344:101::d]:42589
36262025-06-03T00:53:40.663ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42589 req_id = a8b1e91e-8972-4c2c-b13f-7a4bd52f131d roles = RoleSet { roles: {} } uri = /metrics/collectors/292a19e1-4305-4e49-b11e-08ac74714bf3/producers?sort_by=id_ascending
36272025-06-03T00:53:40.663ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42589 req_id = a8b1e91e-8972-4c2c-b13f-7a4bd52f131d resource = Database result = Ok(()) uri = /metrics/collectors/292a19e1-4305-4e49-b11e-08ac74714bf3/producers?sort_by=id_ascending
36282025-06-03T00:53:40.666ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2941 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42589 req_id = a8b1e91e-8972-4c2c-b13f-7a4bd52f131d response_code = 200 uri = /metrics/collectors/292a19e1-4305-4e49-b11e-08ac74714bf3/producers?sort_by=id_ascending
36292025-06-03T00:53:40.666ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42589 req_id = fea3dc48-eb16-4c68-92e4-31674d662dcf roles = RoleSet { roles: {} } uri = /metrics/collectors/292a19e1-4305-4e49-b11e-08ac74714bf3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmZTEyZmZjMS1mMmFjLTRiZjMtOTY3Mi1mYzE4NWY4MmM1OWIifX0%3D
36302025-06-03T00:53:40.666ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42589 req_id = fea3dc48-eb16-4c68-92e4-31674d662dcf resource = Database result = Ok(()) uri = /metrics/collectors/292a19e1-4305-4e49-b11e-08ac74714bf3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmZTEyZmZjMS1mMmFjLTRiZjMtOTY3Mi1mYzE4NWY4MmM1OWIifX0%3D
36312025-06-03T00:53:40.672ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 6164 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42589 req_id = fea3dc48-eb16-4c68-92e4-31674d662dcf response_code = 200 uri = /metrics/collectors/292a19e1-4305-4e49-b11e-08ac74714bf3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmZTEyZmZjMS1mMmFjLTRiZjMtOTY3Mi1mYzE4NWY4MmM1OWIifX0%3D
36322025-06-03T00:53:40.680ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
36332025-06-03T00:53:40.680ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
36342025-06-03T00:53:40.680ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1187 zone_address = fd00:1122:3344:101::2
36352025-06-03T00:53:40.680ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
36362025-06-03T00:53:40.681ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "73fc3b6c-689d-4341-820f-ecce948dc3f7", "content-length": "26", "date": "Tue, 03 Jun 2025 00:53:40 GMT"} })
36372025-06-03T00:53:40.681ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1190 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
36382025-06-03T00:53:40.682ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 175.732967ms iteration = 7 status = {"physical_disks_added":0}
36392025-06-03T00:53:40.682ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1224 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
36402025-06-03T00:53:40.757ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 250.297652ms iteration = 6 status = {}
36412025-06-03T00:53:40.767ZINFOnexus (BackgroundTasks): VPC route manager sled afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
36422025-06-03T00:53:40.767ZDEBGnexus (BackgroundTasks): client request SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
36432025-06-03T00:53:40.788ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
36442025-06-03T00:53:40.788ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
36452025-06-03T00:53:40.789ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
36462025-06-03T00:53:40.789ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
36472025-06-03T00:53:40.790ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: ec5e33f9-c697-46e2-a93e-a745336bf9dc, rack_id: d6831ba0-55f0-4191-8dc2-ea15e55ab0ae, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(9378a284-0ee9-48e2-80a0-3bcaadb1c4e2) } rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae switch_location = Switch0
36482025-06-03T00:53:40.790ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
36492025-06-03T00:53:40.791ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
36502025-06-03T00:53:40.794ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36512025-06-03T00:53:40.796ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
36522025-06-03T00:53:40.796ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
36532025-06-03T00:53:40.796ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
36542025-06-03T00:53:40.797ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36552025-06-03T00:53:40.799ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
36562025-06-03T00:53:40.799ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
36572025-06-03T00:53:40.799ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
36582025-06-03T00:53:40.804ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1832 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
36592025-06-03T00:53:40.804ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1859 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 9378a284-0ee9-48e2-80a0-3bcaadb1c4e2
36602025-06-03T00:53:40.804ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
36612025-06-03T00:53:40.804ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
36622025-06-03T00:53:40.804ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
36632025-06-03T00:53:40.804ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
36642025-06-03T00:53:40.804ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
36652025-06-03T00:53:40.804ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2006 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
36662025-06-03T00:53:40.804ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:408 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
36672025-06-03T00:53:40.804ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2042 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
36682025-06-03T00:53:40.805ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae
36692025-06-03T00:53:40.805ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
36702025-06-03T00:53:40.805ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
36712025-06-03T00:53:40.808ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae switch_location = Switch0
36722025-06-03T00:53:40.808ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
36732025-06-03T00:53:40.817ZDEBGnexus (BackgroundTasks): client response SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "e58b96af-d5e9-40ab-9bf9-4f49a0775402", "content-length": "372", "date": "Tue, 03 Jun 2025 00:53:40 GMT"} })
36742025-06-03T00:53:40.817ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
36752025-06-03T00:53:40.818ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
36762025-06-03T00:53:40.869ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
36772025-06-03T00:53:40.869ZDEBGnexus (BackgroundTasks): client request SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
36782025-06-03T00:53:40.870ZDEBGnexus (BackgroundTasks): client response SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "4c036417-a3fb-4fdb-a521-04657504b4e6", "date": "Tue, 03 Jun 2025 00:53:40 GMT"} })
36792025-06-03T00:53:40.870ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
36802025-06-03T00:53:40.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
36812025-06-03T00:53:40.873ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
36822025-06-03T00:53:40.873ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
36832025-06-03T00:53:40.876ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
36842025-06-03T00:53:40.876ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
36852025-06-03T00:53:40.897ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
36862025-06-03T00:53:40.897ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
36872025-06-03T00:53:40.903ZDEBGnexus (BackgroundTasks): client request SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
36882025-06-03T00:53:40.906ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
36892025-06-03T00:53:40.906ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
36902025-06-03T00:53:40.907ZINFOnexus (BackgroundTasks): VPC route manager sled afe89ed7-19b7-479b-81eb-05c988ebc66d push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
36912025-06-03T00:53:40.907ZDEBGnexus (BackgroundTasks): client request SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
36922025-06-03T00:53:40.907ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36932025-06-03T00:53:40.911ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: afe89ed7-19b7-479b-81eb-05c988ebc66d, lookup_type: ById(afe89ed7-19b7-479b-81eb-05c988ebc66d) } result = Ok(())
36942025-06-03T00:53:40.911ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
36952025-06-03T00:53:40.911ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
36962025-06-03T00:53:40.913ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 406.638647ms iteration = 6 status = 0
36972025-06-03T00:53:40.917ZDEBGnexus (BackgroundTasks): client response SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "e620edf7-e984-4cbb-9c69-429a341f1e6b", "content-length": "691", "date": "Tue, 03 Jun 2025 00:53:40 GMT"} })
36982025-06-03T00:53:40.917ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
36992025-06-03T00:53:40.917ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
37002025-06-03T00:53:40.917ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
37012025-06-03T00:53:40.917ZDEBGnexus (BackgroundTasks): client response SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "ffa2251f-7ad9-4de9-bc67-dd8a442a339c", "date": "Tue, 03 Jun 2025 00:53:40 GMT"} })
37022025-06-03T00:53:40.918ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 411.863359ms iteration = 6 status = {}
37032025-06-03T00:53:40.918ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 411.955662ms iteration = 6 status = {}
37042025-06-03T00:53:40.933ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "f64cea75-b04b-439a-885d-41c54e1ccf3c", "date": "Tue, 03 Jun 2025 00:53:40 GMT"} })
37052025-06-03T00:53:40.933ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae switch = Switch0
37062025-06-03T00:53:40.933ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
37072025-06-03T00:53:40.938ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "e23b939a-8e77-4f8a-84bf-4b95b7d7ec0e", "content-length": "682", "date": "Tue, 03 Jun 2025 00:53:40 GMT"} })
37082025-06-03T00:53:40.938ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
37092025-06-03T00:53:40.938ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
37102025-06-03T00:53:40.941ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
37112025-06-03T00:53:40.941ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
37122025-06-03T00:53:40.945ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
37132025-06-03T00:53:40.945ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
37142025-06-03T00:53:40.947ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
37152025-06-03T00:53:40.947ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
37162025-06-03T00:53:40.955ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 448.590259ms iteration = 6 status = {}
37172025-06-03T00:53:45.675ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 337 local_addr = [fd00:1122:3344:101::a]:63576 method = GET remote_addr = [fd00:1122:3344:101::d]:61315 req_id = 4ceb3b46-1930-4194-8702-56536f50594c response_code = 200 uri = /a2861392-5fd2-462b-84e5-6f36c716263a
37182025-06-03T00:53:50.506ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 17 reason = Timeout
37192025-06-03T00:53:50.506ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 17 reason = Timeout
37202025-06-03T00:53:50.506ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
37212025-06-03T00:53:50.506ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
37222025-06-03T00:53:50.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
37232025-06-03T00:53:50.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
37242025-06-03T00:53:50.508ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37252025-06-03T00:53:50.510ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
37262025-06-03T00:53:50.510ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
37272025-06-03T00:53:50.511ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
37282025-06-03T00:53:50.513ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
37292025-06-03T00:53:50.513ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
37302025-06-03T00:53:50.515ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37312025-06-03T00:53:50.517ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 4ec93699-353f-4b9e-ae06-25355fdb069e, lookup_type: ById(4ec93699-353f-4b9e-ae06-25355fdb069e) } result = Ok(())
37322025-06-03T00:53:50.517ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
37332025-06-03T00:53:50.517ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
37342025-06-03T00:53:50.530ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 4ec93699-353f-4b9e-ae06-25355fdb069e original_time_created = 2025-06-03 00:51:07.059326 UTC target_id = 4ec93699-353f-4b9e-ae06-25355fdb069e time_created = 2025-06-03 00:51:07.059326 UTC
37352025-06-03T00:53:50.530ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 24.007426ms iteration = 17 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"4ec93699-353f-4b9e-ae06-25355fdb069e","time_created":"2025-06-03T00:51:07.059326Z"}
37362025-06-03T00:53:50.539ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
37372025-06-03T00:53:50.539ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
37382025-06-03T00:53:50.540ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 34.711818ms iteration = 17 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
37392025-06-03T00:53:55.674ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 405 local_addr = [fd00:1122:3344:101::a]:63576 method = GET remote_addr = [fd00:1122:3344:101::d]:61315 req_id = b964fa4f-4995-4a1e-808f-a3fabdf7fa68 response_code = 200 uri = /a2861392-5fd2-462b-84e5-6f36c716263a
37402025-06-03T00:54:00.506ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 18 reason = Timeout
37412025-06-03T00:54:00.506ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
37422025-06-03T00:54:00.506ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 18 reason = Timeout
37432025-06-03T00:54:00.506ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
37442025-06-03T00:54:00.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
37452025-06-03T00:54:00.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
37462025-06-03T00:54:00.508ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
37472025-06-03T00:54:00.509ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
37482025-06-03T00:54:00.509ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37492025-06-03T00:54:00.510ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.889081ms iteration = 18 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
37502025-06-03T00:54:00.512ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
37512025-06-03T00:54:00.512ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
37522025-06-03T00:54:00.512ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
37532025-06-03T00:54:00.514ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
37542025-06-03T00:54:00.514ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
37552025-06-03T00:54:00.516ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37562025-06-03T00:54:00.518ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 4ec93699-353f-4b9e-ae06-25355fdb069e, lookup_type: ById(4ec93699-353f-4b9e-ae06-25355fdb069e) } result = Ok(())
37572025-06-03T00:54:00.518ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
37582025-06-03T00:54:00.519ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
37592025-06-03T00:54:00.531ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 4ec93699-353f-4b9e-ae06-25355fdb069e original_time_created = 2025-06-03 00:51:07.059326 UTC target_id = 4ec93699-353f-4b9e-ae06-25355fdb069e time_created = 2025-06-03 00:51:07.059326 UTC
37602025-06-03T00:54:00.531ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 24.856152ms iteration = 18 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"4ec93699-353f-4b9e-ae06-25355fdb069e","time_created":"2025-06-03T00:51:07.059326Z"}
37612025-06-03T00:54:05.674ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 312 local_addr = [fd00:1122:3344:101::a]:63576 method = GET remote_addr = [fd00:1122:3344:101::d]:61315 req_id = d29989bc-190d-4767-b466-e4390b267439 response_code = 200 uri = /a2861392-5fd2-462b-84e5-6f36c716263a
37622025-06-03T00:54:10.505ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 4 reason = Timeout
37632025-06-03T00:54:10.505ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
37642025-06-03T00:54:10.505ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
37652025-06-03T00:54:10.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
37662025-06-03T00:54:10.506ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 19 reason = Timeout
37672025-06-03T00:54:10.506ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
37682025-06-03T00:54:10.506ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 7 reason = Timeout
37692025-06-03T00:54:10.506ZDEBGnexus (BackgroundTasks): activating background_task = alert_dispatcher iteration = 4 reason = Timeout
37702025-06-03T00:54:10.506ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
37712025-06-03T00:54:10.506ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
37722025-06-03T00:54:10.506ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 4 reason = Timeout
37732025-06-03T00:54:10.506ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
37742025-06-03T00:54:10.506ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
37752025-06-03T00:54:10.506ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 7 reason = Timeout
37762025-06-03T00:54:10.506ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 4 reason = Timeout
37772025-06-03T00:54:10.506ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:53
37782025-06-03T00:54:10.506ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 4 reason = Timeout
37792025-06-03T00:54:10.506ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
37802025-06-03T00:54:10.506ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
37812025-06-03T00:54:10.506ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 4 reason = Timeout
37822025-06-03T00:54:10.506ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-06-03 00:44:10.506267140 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
37832025-06-03T00:54:10.506ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
37842025-06-03T00:54:10.506ZDEBGnexus (BackgroundTasks): activating background_task = webhook_deliverator iteration = 4 reason = Timeout
37852025-06-03T00:54:10.506ZDEBGnexus (BackgroundTasks): roles background_task = webhook_deliverator roles = RoleSet { roles: {} }
37862025-06-03T00:54:10.506ZDEBGnexus (BackgroundTasks): activating background_task = crdb_node_id_collector iteration = 3 reason = Timeout
37872025-06-03T00:54:10.506ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 6 reason = Timeout
37882025-06-03T00:54:10.506ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
37892025-06-03T00:54:10.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
37902025-06-03T00:54:10.506ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 4 reason = Timeout
37912025-06-03T00:54:10.506ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 7 reason = Timeout
37922025-06-03T00:54:10.506ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
37932025-06-03T00:54:10.506ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
37942025-06-03T00:54:10.506ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 6 reason = Timeout
37952025-06-03T00:54:10.506ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
37962025-06-03T00:54:10.506ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 4 reason = Timeout
37972025-06-03T00:54:10.506ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
37982025-06-03T00:54:10.506ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 7 reason = Timeout
37992025-06-03T00:54:10.506ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
38002025-06-03T00:54:10.506ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 7 reason = Timeout
38012025-06-03T00:54:10.506ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
38022025-06-03T00:54:10.506ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 19 reason = Timeout
38032025-06-03T00:54:10.506ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 7 reason = Timeout
38042025-06-03T00:54:10.506ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
38052025-06-03T00:54:10.506ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 8 reason = Timeout
38062025-06-03T00:54:10.506ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
38072025-06-03T00:54:10.506ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 7 reason = Timeout
38082025-06-03T00:54:10.506ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
38092025-06-03T00:54:10.506ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
38102025-06-03T00:54:10.506ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 7 reason = Timeout
38112025-06-03T00:54:10.506ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
38122025-06-03T00:54:10.506ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 7 reason = Timeout
38132025-06-03T00:54:10.506ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
38142025-06-03T00:54:10.506ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 7 reason = Timeout
38152025-06-03T00:54:10.506ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
38162025-06-03T00:54:10.506ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 4 reason = Timeout
38172025-06-03T00:54:10.506ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
38182025-06-03T00:54:10.506ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 4 reason = Timeout
38192025-06-03T00:54:10.506ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
38202025-06-03T00:54:10.506ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 7 reason = Timeout
38212025-06-03T00:54:10.506ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
38222025-06-03T00:54:10.506ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
38232025-06-03T00:54:10.506ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 7 reason = Timeout
38242025-06-03T00:54:10.506ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
38252025-06-03T00:54:10.506ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 7 reason = Timeout
38262025-06-03T00:54:10.506ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
38272025-06-03T00:54:10.506ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 7 reason = Timeout
38282025-06-03T00:54:10.506ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
38292025-06-03T00:54:10.506ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 7 reason = Timeout
38302025-06-03T00:54:10.506ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 7 reason = Timeout
38312025-06-03T00:54:10.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
38322025-06-03T00:54:10.506ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
38332025-06-03T00:54:10.507ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 5 reason = Timeout
38342025-06-03T00:54:10.507ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:85 reason = blueprint disabled target_id = 4ec93699-353f-4b9e-ae06-25355fdb069e
38352025-06-03T00:54:10.507ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 4 reason = Timeout
38362025-06-03T00:54:10.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = webhook_deliverator resource = Database result = Ok(())
38372025-06-03T00:54:10.507ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 11.51\u{b5}s iteration = 5 status = {"enabled":false,"target_id":"4ec93699-353f-4b9e-ae06-25355fdb069e"}
38382025-06-03T00:54:10.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
38392025-06-03T00:54:10.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
38402025-06-03T00:54:10.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
38412025-06-03T00:54:10.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
38422025-06-03T00:54:10.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = alert_dispatcher resource = Database result = Ok(())
38432025-06-03T00:54:10.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
38442025-06-03T00:54:10.507ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
38452025-06-03T00:54:10.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
38462025-06-03T00:54:10.507ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
38472025-06-03T00:54:10.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
38482025-06-03T00:54:10.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
38492025-06-03T00:54:10.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
38502025-06-03T00:54:10.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
38512025-06-03T00:54:10.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
38522025-06-03T00:54:10.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
38532025-06-03T00:54:10.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
38542025-06-03T00:54:10.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
38552025-06-03T00:54:10.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
38562025-06-03T00:54:10.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
38572025-06-03T00:54:10.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
38582025-06-03T00:54:10.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
38592025-06-03T00:54:10.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
38602025-06-03T00:54:10.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
38612025-06-03T00:54:10.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
38622025-06-03T00:54:10.507ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
38632025-06-03T00:54:10.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
38642025-06-03T00:54:10.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
38652025-06-03T00:54:10.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
38662025-06-03T00:54:10.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
38672025-06-03T00:54:10.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
38682025-06-03T00:54:10.507ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("241573bc-467c-4802-9f4b-f7c82da4e12e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("64fc0a69-d4b8-4f6c-9a07-77a4cfa94962.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("ceb74144-4b6b-4b3f-bc80-eac2d75f3bc9.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("ceb74144-4b6b-4b3f-bc80-eac2d75f3bc9.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:2::1))) }], valid_until: Instant { tv_sec: 2577, tv_nsec: 731204265 } })
38692025-06-03T00:54:10.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
38702025-06-03T00:54:10.507ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
38712025-06-03T00:54:10.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
38722025-06-03T00:54:10.507ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
38732025-06-03T00:54:10.507ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("9feb16c9-ff60-43e8-917b-965124f46e0d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("aa124775-31de-4648-981a-4af500268b66.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("aa124775-31de-4648-981a-4af500268b66.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::9))) }], valid_until: Instant { tv_sec: 2577, tv_nsec: 731395031 } })
38742025-06-03T00:54:10.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
38752025-06-03T00:54:10.508ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
38762025-06-03T00:54:10.508ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 2.098595ms iteration = 4 status = {"addresses":["[fd00:1122:3344:101::8]:5353","[fd00:1122:3344:101::9]:5353"]}
38772025-06-03T00:54:10.508ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } dns_group = internal
38782025-06-03T00:54:10.508ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.594ms iteration = 4 status = {"addresses":["[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:3::1]:5353","[fd00:1122:3344:2::1]:5353"]}
38792025-06-03T00:54:10.510ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38802025-06-03T00:54:10.512ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
38812025-06-03T00:54:10.512ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
38822025-06-03T00:54:10.513ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
38832025-06-03T00:54:10.513ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
38842025-06-03T00:54:10.514ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
38852025-06-03T00:54:10.517ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38862025-06-03T00:54:10.519ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38872025-06-03T00:54:10.519ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
38882025-06-03T00:54:10.519ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
38892025-06-03T00:54:10.520ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
38902025-06-03T00:54:10.520ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-06-03 00:44:10.506267140 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
38912025-06-03T00:54:10.520ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 14.189111ms iteration = 4 status = {"expiration":"2025-06-03T00:44:10.506267140Z","pruned":[]}
38922025-06-03T00:54:10.521ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38932025-06-03T00:54:10.522ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
38942025-06-03T00:54:10.522ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
38952025-06-03T00:54:10.522ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38962025-06-03T00:54:10.522ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
38972025-06-03T00:54:10.523ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 17.011118ms iteration = 7 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
38982025-06-03T00:54:10.524ZINFOnexus (BackgroundTasks): all webhook delivery tasks completed background_task = webhook_deliverator file = nexus/src/app/background/tasks/webhook_deliverator.rs:202 num_receivers = 0
38992025-06-03T00:54:10.524ZDEBGnexus (BackgroundTasks): activation complete background_task = webhook_deliverator elapsed = 18.263957ms iteration = 4 status = {"by_rx":{},"error":null}
39002025-06-03T00:54:10.525ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
39012025-06-03T00:54:10.525ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
39022025-06-03T00:54:10.525ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
39032025-06-03T00:54:10.525ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
39042025-06-03T00:54:10.525ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
39052025-06-03T00:54:10.526ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
39062025-06-03T00:54:10.531ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
39072025-06-03T00:54:10.531ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
39082025-06-03T00:54:10.532ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
39092025-06-03T00:54:10.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
39102025-06-03T00:54:10.533ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39112025-06-03T00:54:10.537ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
39122025-06-03T00:54:10.537ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
39132025-06-03T00:54:10.537ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
39142025-06-03T00:54:10.540ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39152025-06-03T00:54:10.540ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39162025-06-03T00:54:10.540ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 33.621964ms iteration = 4 status = {"errors":[],"found_port_ok":[]}
39172025-06-03T00:54:10.541ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-afe89ed7-19b7-479b-81eb-05c988ebc66d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-afe89ed7-19b7-479b-81eb-05c988ebc66d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 2577, tv_nsec: 764948322 } })
39182025-06-03T00:54:10.541ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1179
39192025-06-03T00:54:10.543ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
39202025-06-03T00:54:10.543ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
39212025-06-03T00:54:10.543ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
39222025-06-03T00:54:10.543ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
39232025-06-03T00:54:10.543ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
39242025-06-03T00:54:10.544ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
39252025-06-03T00:54:10.545ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-afe89ed7-19b7-479b-81eb-05c988ebc66d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-afe89ed7-19b7-479b-81eb-05c988ebc66d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 2577, tv_nsec: 769369800 } })
39262025-06-03T00:54:10.546ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1179
39272025-06-03T00:54:10.549ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39282025-06-03T00:54:10.550ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39292025-06-03T00:54:10.552ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
39302025-06-03T00:54:10.552ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
39312025-06-03T00:54:10.554ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
39322025-06-03T00:54:10.554ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
39332025-06-03T00:54:10.554ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
39342025-06-03T00:54:10.554ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
39352025-06-03T00:54:10.554ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39362025-06-03T00:54:10.557ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
39372025-06-03T00:54:10.557ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
39382025-06-03T00:54:10.557ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
39392025-06-03T00:54:10.559ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39402025-06-03T00:54:10.561ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
39412025-06-03T00:54:10.562ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
39422025-06-03T00:54:10.562ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
39432025-06-03T00:54:10.564ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39442025-06-03T00:54:10.564ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
39452025-06-03T00:54:10.564ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
39462025-06-03T00:54:10.568ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
39472025-06-03T00:54:10.572ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39482025-06-03T00:54:10.575ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
39492025-06-03T00:54:10.575ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
39502025-06-03T00:54:10.575ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
39512025-06-03T00:54:10.575ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
39522025-06-03T00:54:10.575ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
39532025-06-03T00:54:10.575ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
39542025-06-03T00:54:10.575ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
39552025-06-03T00:54:10.576ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
39562025-06-03T00:54:10.576ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-afe89ed7-19b7-479b-81eb-05c988ebc66d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-afe89ed7-19b7-479b-81eb-05c988ebc66d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 2577, tv_nsec: 800259309 } })
39572025-06-03T00:54:10.576ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1179 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae
39582025-06-03T00:54:10.578ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 71.543041ms iteration = 19 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
39592025-06-03T00:54:10.579ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
39602025-06-03T00:54:10.580ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
39612025-06-03T00:54:10.580ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:547 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
39622025-06-03T00:54:10.580ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 74.208064ms iteration = 7 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
39632025-06-03T00:54:10.581ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
39642025-06-03T00:54:10.581ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
39652025-06-03T00:54:10.582ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
39662025-06-03T00:54:10.582ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
39672025-06-03T00:54:10.598ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 92.241164ms iteration = 7 status = {"errors":[],"garbage_collect_requested":[]}
39682025-06-03T00:54:10.598ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39692025-06-03T00:54:10.601ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
39702025-06-03T00:54:10.601ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
39712025-06-03T00:54:10.602ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
39722025-06-03T00:54:10.610ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
39732025-06-03T00:54:10.610ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39742025-06-03T00:54:10.611ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
39752025-06-03T00:54:10.611ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39762025-06-03T00:54:10.613ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 107.57356ms iteration = 7 status = {"errors":[],"finish_invoked_ok":[]}
39772025-06-03T00:54:10.614ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
39782025-06-03T00:54:10.614ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
39792025-06-03T00:54:10.615ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
39802025-06-03T00:54:10.615ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 4ec93699-353f-4b9e-ae06-25355fdb069e, lookup_type: ById(4ec93699-353f-4b9e-ae06-25355fdb069e) } result = Ok(())
39812025-06-03T00:54:10.615ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
39822025-06-03T00:54:10.615ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39832025-06-03T00:54:10.615ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
39842025-06-03T00:54:10.617ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39852025-06-03T00:54:10.619ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
39862025-06-03T00:54:10.619ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
39872025-06-03T00:54:10.619ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
39882025-06-03T00:54:10.619ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39892025-06-03T00:54:10.619ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
39902025-06-03T00:54:10.619ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
39912025-06-03T00:54:10.621ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
39922025-06-03T00:54:10.621ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
39932025-06-03T00:54:10.621ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
39942025-06-03T00:54:10.623ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
39952025-06-03T00:54:10.623ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:167 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
39962025-06-03T00:54:10.623ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 116.850749ms iteration = 4 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
39972025-06-03T00:54:10.624ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
39982025-06-03T00:54:10.624ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
39992025-06-03T00:54:10.624ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
40002025-06-03T00:54:10.627ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
40012025-06-03T00:54:10.628ZDEBGnexus (BackgroundTasks): activation complete background_task = crdb_node_id_collector elapsed = 121.866644ms iteration = 3 status = {"nsuccess":5}
40022025-06-03T00:54:10.628ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
40032025-06-03T00:54:10.628ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
40042025-06-03T00:54:10.629ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 122.689459ms iteration = 7 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
40052025-06-03T00:54:10.631ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40062025-06-03T00:54:10.632ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
40072025-06-03T00:54:10.632ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
40082025-06-03T00:54:10.632ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
40092025-06-03T00:54:10.632ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
40102025-06-03T00:54:10.632ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
40112025-06-03T00:54:10.634ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
40122025-06-03T00:54:10.634ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
40132025-06-03T00:54:10.634ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
40142025-06-03T00:54:10.636ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40152025-06-03T00:54:10.638ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
40162025-06-03T00:54:10.638ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
40172025-06-03T00:54:10.638ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
40182025-06-03T00:54:10.639ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
40192025-06-03T00:54:10.639ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 4ec93699-353f-4b9e-ae06-25355fdb069e original_time_created = 2025-06-03 00:51:07.059326 UTC target_id = 4ec93699-353f-4b9e-ae06-25355fdb069e time_created = 2025-06-03 00:51:07.059326 UTC
40202025-06-03T00:54:10.639ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 133.862316ms iteration = 19 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"4ec93699-353f-4b9e-ae06-25355fdb069e","time_created":"2025-06-03T00:51:07.059326Z"}
40212025-06-03T00:54:10.640ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
40222025-06-03T00:54:10.641ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
40232025-06-03T00:54:10.641ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
40242025-06-03T00:54:10.641ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
40252025-06-03T00:54:10.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
40262025-06-03T00:54:10.642ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 135.945332ms iteration = 7 status = {"errors":[],"requests_created_ok":[]}
40272025-06-03T00:54:10.644ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
40282025-06-03T00:54:10.644ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
40292025-06-03T00:54:10.644ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
40302025-06-03T00:54:10.645ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 138.16102ms iteration = 7 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
40312025-06-03T00:54:10.645ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
40322025-06-03T00:54:10.645ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
40332025-06-03T00:54:10.645ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 9e79f9b1-a4de-4f9d-a901-16cc4f79e15d dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
40342025-06-03T00:54:10.645ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
40352025-06-03T00:54:10.646ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
40362025-06-03T00:54:10.647ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
40372025-06-03T00:54:10.647ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
40382025-06-03T00:54:10.648ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40392025-06-03T00:54:10.650ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40402025-06-03T00:54:10.652ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
40412025-06-03T00:54:10.652ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
40422025-06-03T00:54:10.653ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
40432025-06-03T00:54:10.654ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
40442025-06-03T00:54:10.654ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
40452025-06-03T00:54:10.654ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
40462025-06-03T00:54:10.656ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
40472025-06-03T00:54:10.663ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42589 req_id = c56973f4-b22d-4dac-9665-3aa9b715a631 roles = RoleSet { roles: {} } uri = /metrics/collectors/292a19e1-4305-4e49-b11e-08ac74714bf3/producers?sort_by=id_ascending
40482025-06-03T00:54:10.664ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42589 req_id = c56973f4-b22d-4dac-9665-3aa9b715a631 resource = Database result = Ok(()) uri = /metrics/collectors/292a19e1-4305-4e49-b11e-08ac74714bf3/producers?sort_by=id_ascending
40492025-06-03T00:54:10.670ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 7040 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42589 req_id = c56973f4-b22d-4dac-9665-3aa9b715a631 response_code = 200 uri = /metrics/collectors/292a19e1-4305-4e49-b11e-08ac74714bf3/producers?sort_by=id_ascending
40502025-06-03T00:54:10.671ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42589 req_id = 7e4566e1-98e8-4ef5-8a21-f51a2c880cfe roles = RoleSet { roles: {} } uri = /metrics/collectors/292a19e1-4305-4e49-b11e-08ac74714bf3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmZTEyZmZjMS1mMmFjLTRiZjMtOTY3Mi1mYzE4NWY4MmM1OWIifX0%3D
40512025-06-03T00:54:10.671ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42589 req_id = 7e4566e1-98e8-4ef5-8a21-f51a2c880cfe resource = Database result = Ok(()) uri = /metrics/collectors/292a19e1-4305-4e49-b11e-08ac74714bf3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmZTEyZmZjMS1mMmFjLTRiZjMtOTY3Mi1mYzE4NWY4MmM1OWIifX0%3D
40522025-06-03T00:54:10.674ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 3356 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42589 req_id = 7e4566e1-98e8-4ef5-8a21-f51a2c880cfe response_code = 200 uri = /metrics/collectors/292a19e1-4305-4e49-b11e-08ac74714bf3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmZTEyZmZjMS1mMmFjLTRiZjMtOTY3Mi1mYzE4NWY4MmM1OWIifX0%3D
40532025-06-03T00:54:10.676ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
40542025-06-03T00:54:10.676ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:137 found = 0 not_ready_to_be_deleted = 0
40552025-06-03T00:54:10.676ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 170.495134ms iteration = 4 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
40562025-06-03T00:54:10.679ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
40572025-06-03T00:54:10.679ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = alert_dispatcher resource = Database result = Ok(())
40582025-06-03T00:54:10.683ZDEBGnexus (BackgroundTasks): activation complete background_task = alert_dispatcher elapsed = 177.070508ms iteration = 4 status = {"dispatched":[],"errors":[],"glob_version":"144.0.0","globs_reprocessed":{},"no_receivers":[]}
40592025-06-03T00:54:10.685ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40602025-06-03T00:54:10.688ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40612025-06-03T00:54:10.690ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
40622025-06-03T00:54:10.690ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
40632025-06-03T00:54:10.690ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
40642025-06-03T00:54:10.690ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 184.038614ms iteration = 7 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
40652025-06-03T00:54:10.691ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
40662025-06-03T00:54:10.691ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
40672025-06-03T00:54:10.691ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
40682025-06-03T00:54:10.694ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
40692025-06-03T00:54:10.694ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
40702025-06-03T00:54:10.697ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40712025-06-03T00:54:10.698ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
40722025-06-03T00:54:10.698ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
40732025-06-03T00:54:10.698ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 192.227669ms iteration = 7 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
40742025-06-03T00:54:10.699ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1187 zone_address = fd00:1122:3344:101::2
40752025-06-03T00:54:10.699ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
40762025-06-03T00:54:10.699ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
40772025-06-03T00:54:10.699ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
40782025-06-03T00:54:10.699ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "d3583307-2186-436a-8499-9a451d43691a", "content-length": "26", "date": "Tue, 03 Jun 2025 00:54:10 GMT"} })
40792025-06-03T00:54:10.699ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1190 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
40802025-06-03T00:54:10.699ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
40812025-06-03T00:54:10.702ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1224 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
40822025-06-03T00:54:10.708ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 201.813526ms iteration = 8 status = {"physical_disks_added":0}
40832025-06-03T00:54:10.710ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:372
40842025-06-03T00:54:10.710ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 204.315754ms iteration = 7 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
40852025-06-03T00:54:10.724ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1187 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae zone_address = fd00:1122:3344:101::2
40862025-06-03T00:54:10.724ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
40872025-06-03T00:54:10.724ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "61f684bc-b2bd-4e49-b84c-74f9a9b98062", "content-length": "26", "date": "Tue, 03 Jun 2025 00:54:10 GMT"} })
40882025-06-03T00:54:10.724ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1190 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
40892025-06-03T00:54:10.726ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1224 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae
40902025-06-03T00:54:10.729ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
40912025-06-03T00:54:10.729ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
40922025-06-03T00:54:10.739ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
40932025-06-03T00:54:10.739ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 3ef110b4-9400-4c37-8d7f-5552c8a94db0 dns_zone_name = oxide.test found_so_far = 0 version = 2
40942025-06-03T00:54:10.739ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
40952025-06-03T00:54:10.739ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
40962025-06-03T00:54:10.741ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40972025-06-03T00:54:10.744ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
40982025-06-03T00:54:10.744ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
40992025-06-03T00:54:10.744ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
41002025-06-03T00:54:10.748ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 3ef110b4-9400-4c37-8d7f-5552c8a94db0 dns_zone_name = oxide.test found_so_far = 4 version = 2
41012025-06-03T00:54:10.748ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
41022025-06-03T00:54:10.748ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2025-06-03 00:51:38.567192 UTC dns_group = external generation = 2
41032025-06-03T00:54:10.748ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 241.838079ms iteration = 4 status = {"generation":2}
41042025-06-03T00:54:10.769ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-06-03T00:51:07.585932Z, time_modified: 2025-06-03T00:51:07.585932Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: ea0ef520-6bf3-4fbd-9ff0-f3698b48b749, db_silo: Silo { identity: SiloIdentity { id: ea0ef520-6bf3-4fbd-9ff0-f3698b48b749, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-06-03T00:51:38.523369Z, time_modified: 2025-06-03T00:51:38.523369Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "bcda8a9f16e63c3cec49f9759478b9f335a661639158b0a9026f69b846d4e127" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: ea0ef520-6bf3-4fbd-9ff0-f3698b48b749, db_silo: Silo { identity: SiloIdentity { id: ea0ef520-6bf3-4fbd-9ff0-f3698b48b749, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-06-03T00:51:38.523369Z, time_modified: 2025-06-03T00:51:38.523369Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "bcda8a9f16e63c3cec49f9759478b9f335a661639158b0a9026f69b846d4e127" }] }) }
41052025-06-03T00:54:10.770ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 263.280985ms iteration = 4 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"ea0ef520-6bf3-4fbd-9ff0-f3698b48b749","tls_certs":["bcda8a9f16e63c3cec49f9759478b9f335a661639158b0a9026f69b846d4e127"]}},"default_endpoint":{"silo_id":"ea0ef520-6bf3-4fbd-9ff0-f3698b48b749","tls_certs":["bcda8a9f16e63c3cec49f9759478b9f335a661639158b0a9026f69b846d4e127"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
41062025-06-03T00:54:10.787ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
41072025-06-03T00:54:10.791ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 9e79f9b1-a4de-4f9d-a901-16cc4f79e15d dns_zone_name = control-plane.oxide.internal found_so_far = 59 version = 1
41082025-06-03T00:54:10.791ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
41092025-06-03T00:54:10.791ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2025-06-03 00:51:38.152773 UTC dns_group = internal generation = 1
41102025-06-03T00:54:10.791ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 286.30144ms iteration = 4 status = {"generation":1}
41112025-06-03T00:54:10.835ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1187 zone_address = fd00:1122:3344:101::2
41122025-06-03T00:54:10.835ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
41132025-06-03T00:54:10.836ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "8e270bd9-a823-4cb4-9c55-b02f0507d93c", "content-length": "26", "date": "Tue, 03 Jun 2025 00:54:10 GMT"} })
41142025-06-03T00:54:10.836ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1190 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
41152025-06-03T00:54:10.838ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1224 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
41162025-06-03T00:54:10.853ZINFOnexus (BackgroundTasks): instance reincarnation completed successfully background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:113 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0
41172025-06-03T00:54:10.853ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 347.345246ms iteration = 4 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
41182025-06-03T00:54:10.903ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
41192025-06-03T00:54:10.904ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
41202025-06-03T00:54:10.905ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 398.69732ms iteration = 7 status = null
41212025-06-03T00:54:10.918ZINFOnexus (BackgroundTasks): VPC route manager sled afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
41222025-06-03T00:54:10.918ZDEBGnexus (BackgroundTasks): client request SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
41232025-06-03T00:54:10.919ZDEBGnexus (BackgroundTasks): client response SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "c280c83e-a19c-4ab3-9967-485db3a8daf9", "content-length": "372", "date": "Tue, 03 Jun 2025 00:54:10 GMT"} })
41242025-06-03T00:54:10.919ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
41252025-06-03T00:54:10.919ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
41262025-06-03T00:54:10.925ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
41272025-06-03T00:54:10.926ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "5af9f2b8-9834-4bb8-be1d-1dbb37d18a89", "date": "Tue, 03 Jun 2025 00:54:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
41282025-06-03T00:54:10.926ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
41292025-06-03T00:54:10.927ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "cabd1068-859c-4076-bdbd-fa471b4c1c27", "date": "Tue, 03 Jun 2025 00:54:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
41302025-06-03T00:54:10.927ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
41312025-06-03T00:54:10.929ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
41322025-06-03T00:54:10.929ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 423.15871ms iteration = 6 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
41332025-06-03T00:54:10.958ZDEBGnexus (BackgroundTasks): client request SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
41342025-06-03T00:54:10.962ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
41352025-06-03T00:54:10.963ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "55589ec6-3f15-49ee-ae8b-76e44c47d2ea", "date": "Tue, 03 Jun 2025 00:54:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
41362025-06-03T00:54:10.963ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
41372025-06-03T00:54:10.964ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "97c00f55-dd9c-4e51-82b6-be9dc00f9587", "date": "Tue, 03 Jun 2025 00:54:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
41382025-06-03T00:54:10.964ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
41392025-06-03T00:54:10.967ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "eab6b677-b89c-415c-9d60-6e233c0879ea", "date": "Tue, 03 Jun 2025 00:54:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
41402025-06-03T00:54:10.967ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
41412025-06-03T00:54:10.968ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
41422025-06-03T00:54:10.968ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 462.520922ms iteration = 6 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
41432025-06-03T00:54:10.971ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
41442025-06-03T00:54:10.971ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
41452025-06-03T00:54:10.974ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
41462025-06-03T00:54:10.974ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 467.447815ms iteration = 7 status = {}
41472025-06-03T00:54:10.974ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
41482025-06-03T00:54:10.976ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41492025-06-03T00:54:10.979ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: afe89ed7-19b7-479b-81eb-05c988ebc66d, lookup_type: ById(afe89ed7-19b7-479b-81eb-05c988ebc66d) } result = Ok(())
41502025-06-03T00:54:10.979ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
41512025-06-03T00:54:10.979ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
41522025-06-03T00:54:10.981ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 475.00084ms iteration = 7 status = 0
41532025-06-03T00:54:11.013ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
41542025-06-03T00:54:11.013ZDEBGnexus (BackgroundTasks): client request SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
41552025-06-03T00:54:11.021ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
41562025-06-03T00:54:11.021ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
41572025-06-03T00:54:11.023ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: ec5e33f9-c697-46e2-a93e-a745336bf9dc, rack_id: d6831ba0-55f0-4191-8dc2-ea15e55ab0ae, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(9378a284-0ee9-48e2-80a0-3bcaadb1c4e2) } rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae switch_location = Switch0
41582025-06-03T00:54:11.023ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
41592025-06-03T00:54:11.024ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
41602025-06-03T00:54:11.041ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1832 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
41612025-06-03T00:54:11.041ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1859 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 9378a284-0ee9-48e2-80a0-3bcaadb1c4e2
41622025-06-03T00:54:11.042ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
41632025-06-03T00:54:11.042ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
41642025-06-03T00:54:11.042ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
41652025-06-03T00:54:11.042ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
41662025-06-03T00:54:11.042ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
41672025-06-03T00:54:11.042ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2006 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
41682025-06-03T00:54:11.042ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:408 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
41692025-06-03T00:54:11.042ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2042 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
41702025-06-03T00:54:11.042ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae
41712025-06-03T00:54:11.042ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
41722025-06-03T00:54:11.043ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
41732025-06-03T00:54:11.045ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae switch_location = Switch0
41742025-06-03T00:54:11.045ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
41752025-06-03T00:54:11.070ZDEBGnexus (BackgroundTasks): client response SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "193c5c99-54fc-4e5b-a1cc-bb0a29f04879", "date": "Tue, 03 Jun 2025 00:54:11 GMT"} })
41762025-06-03T00:54:11.070ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
41772025-06-03T00:54:11.070ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
41782025-06-03T00:54:11.073ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
41792025-06-03T00:54:11.074ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
41802025-06-03T00:54:11.076ZDEBGnexus (BackgroundTasks): client response SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "a6ec43e8-60db-4f38-b915-eaa57f16841b", "content-length": "691", "date": "Tue, 03 Jun 2025 00:54:11 GMT"} })
41812025-06-03T00:54:11.076ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
41822025-06-03T00:54:11.076ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
41832025-06-03T00:54:11.076ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
41842025-06-03T00:54:11.078ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 572.003372ms iteration = 7 status = {}
41852025-06-03T00:54:11.184ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
41862025-06-03T00:54:11.185ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
41872025-06-03T00:54:11.189ZINFOnexus (BackgroundTasks): VPC route manager sled afe89ed7-19b7-479b-81eb-05c988ebc66d push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
41882025-06-03T00:54:11.189ZDEBGnexus (BackgroundTasks): client request SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
41892025-06-03T00:54:11.195ZDEBGnexus (BackgroundTasks): client response SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "585bbd3d-a7d1-4950-88da-0a15c072380b", "date": "Tue, 03 Jun 2025 00:54:11 GMT"} })
41902025-06-03T00:54:11.197ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 690.858401ms iteration = 7 status = {}
41912025-06-03T00:54:11.206ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "4c492fcd-4d32-4664-a1c5-59103b4693af", "date": "Tue, 03 Jun 2025 00:54:11 GMT"} })
41922025-06-03T00:54:11.206ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae switch = Switch0
41932025-06-03T00:54:11.206ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
41942025-06-03T00:54:11.212ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "4dc94b75-b944-4f74-a8f7-d6cf897e4bfd", "content-length": "682", "date": "Tue, 03 Jun 2025 00:54:11 GMT"} })
41952025-06-03T00:54:11.212ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
41962025-06-03T00:54:11.212ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
41972025-06-03T00:54:11.217ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
41982025-06-03T00:54:11.217ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
41992025-06-03T00:54:11.223ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
42002025-06-03T00:54:11.223ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
42012025-06-03T00:54:11.225ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
42022025-06-03T00:54:11.225ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
42032025-06-03T00:54:11.233ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 727.440977ms iteration = 7 status = {}
42042025-06-03T00:54:15.674ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 286 local_addr = [fd00:1122:3344:101::a]:63576 method = GET remote_addr = [fd00:1122:3344:101::d]:61315 req_id = 3168753f-8961-40b0-ab2b-11fed2cb38b4 response_code = 200 uri = /a2861392-5fd2-462b-84e5-6f36c716263a
42052025-06-03T00:54:20.506ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 20 reason = Timeout
42062025-06-03T00:54:20.506ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 20 reason = Timeout
42072025-06-03T00:54:20.506ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
42082025-06-03T00:54:20.506ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
42092025-06-03T00:54:20.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
42102025-06-03T00:54:20.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
42112025-06-03T00:54:20.508ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
42122025-06-03T00:54:20.509ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
42132025-06-03T00:54:20.509ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42142025-06-03T00:54:20.510ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.48976ms iteration = 20 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
42152025-06-03T00:54:20.511ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
42162025-06-03T00:54:20.511ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
42172025-06-03T00:54:20.512ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
42182025-06-03T00:54:20.514ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
42192025-06-03T00:54:20.514ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
42202025-06-03T00:54:20.516ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42212025-06-03T00:54:20.518ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 4ec93699-353f-4b9e-ae06-25355fdb069e, lookup_type: ById(4ec93699-353f-4b9e-ae06-25355fdb069e) } result = Ok(())
42222025-06-03T00:54:20.518ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
42232025-06-03T00:54:20.518ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
42242025-06-03T00:54:20.526ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 4ec93699-353f-4b9e-ae06-25355fdb069e original_time_created = 2025-06-03 00:51:07.059326 UTC target_id = 4ec93699-353f-4b9e-ae06-25355fdb069e time_created = 2025-06-03 00:51:07.059326 UTC
42252025-06-03T00:54:20.526ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 20.198428ms iteration = 20 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"4ec93699-353f-4b9e-ae06-25355fdb069e","time_created":"2025-06-03T00:51:07.059326Z"}
42262025-06-03T00:54:25.663ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42589 req_id = 6f8679d1-a4e1-4c71-938b-786c5541bada roles = RoleSet { roles: {} } uri = /metrics/collectors/292a19e1-4305-4e49-b11e-08ac74714bf3/producers?sort_by=id_ascending
42272025-06-03T00:54:25.663ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42589 req_id = 6f8679d1-a4e1-4c71-938b-786c5541bada resource = Database result = Ok(()) uri = /metrics/collectors/292a19e1-4305-4e49-b11e-08ac74714bf3/producers?sort_by=id_ascending
42282025-06-03T00:54:25.666ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 3682 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42589 req_id = 6f8679d1-a4e1-4c71-938b-786c5541bada response_code = 200 uri = /metrics/collectors/292a19e1-4305-4e49-b11e-08ac74714bf3/producers?sort_by=id_ascending
42292025-06-03T00:54:25.666ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42589 req_id = 225e9458-4712-489e-8d16-834facb661bf roles = RoleSet { roles: {} } uri = /metrics/collectors/292a19e1-4305-4e49-b11e-08ac74714bf3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmZTEyZmZjMS1mMmFjLTRiZjMtOTY3Mi1mYzE4NWY4MmM1OWIifX0%3D
42302025-06-03T00:54:25.667ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42589 req_id = 225e9458-4712-489e-8d16-834facb661bf resource = Database result = Ok(()) uri = /metrics/collectors/292a19e1-4305-4e49-b11e-08ac74714bf3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmZTEyZmZjMS1mMmFjLTRiZjMtOTY3Mi1mYzE4NWY4MmM1OWIifX0%3D
42312025-06-03T00:54:25.669ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2881 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42589 req_id = 225e9458-4712-489e-8d16-834facb661bf response_code = 200 uri = /metrics/collectors/292a19e1-4305-4e49-b11e-08ac74714bf3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmZTEyZmZjMS1mMmFjLTRiZjMtOTY3Mi1mYzE4NWY4MmM1OWIifX0%3D
42322025-06-03T00:54:25.674ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 258 local_addr = [fd00:1122:3344:101::a]:63576 method = GET remote_addr = [fd00:1122:3344:101::d]:61315 req_id = ee1e46ef-a7f4-4d5c-a526-1912d59a4617 response_code = 200 uri = /a2861392-5fd2-462b-84e5-6f36c716263a
42332025-06-03T00:54:30.506ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 21 reason = Timeout
42342025-06-03T00:54:30.506ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 21 reason = Timeout
42352025-06-03T00:54:30.506ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
42362025-06-03T00:54:30.506ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
42372025-06-03T00:54:30.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
42382025-06-03T00:54:30.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
42392025-06-03T00:54:30.508ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42402025-06-03T00:54:30.509ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
42412025-06-03T00:54:30.509ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
42422025-06-03T00:54:30.511ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.650094ms iteration = 21 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
42432025-06-03T00:54:30.511ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
42442025-06-03T00:54:30.511ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
42452025-06-03T00:54:30.511ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
42462025-06-03T00:54:30.513ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
42472025-06-03T00:54:30.513ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
42482025-06-03T00:54:30.515ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42492025-06-03T00:54:30.517ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 4ec93699-353f-4b9e-ae06-25355fdb069e, lookup_type: ById(4ec93699-353f-4b9e-ae06-25355fdb069e) } result = Ok(())
42502025-06-03T00:54:30.517ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
42512025-06-03T00:54:30.518ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
42522025-06-03T00:54:30.526ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 4ec93699-353f-4b9e-ae06-25355fdb069e original_time_created = 2025-06-03 00:51:07.059326 UTC target_id = 4ec93699-353f-4b9e-ae06-25355fdb069e time_created = 2025-06-03 00:51:07.059326 UTC
42532025-06-03T00:54:30.526ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 20.520297ms iteration = 21 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"4ec93699-353f-4b9e-ae06-25355fdb069e","time_created":"2025-06-03T00:51:07.059326Z"}
42542025-06-03T00:54:35.674ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 397 local_addr = [fd00:1122:3344:101::a]:63576 method = GET remote_addr = [fd00:1122:3344:101::d]:61315 req_id = 6d543380-075e-47f1-9a12-515364b762d8 response_code = 200 uri = /a2861392-5fd2-462b-84e5-6f36c716263a
42552025-06-03T00:54:40.506ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 22 reason = Timeout
42562025-06-03T00:54:40.506ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 8 reason = Timeout
42572025-06-03T00:54:40.506ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
42582025-06-03T00:54:40.506ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
42592025-06-03T00:54:40.506ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 8 reason = Timeout
42602025-06-03T00:54:40.506ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
42612025-06-03T00:54:40.506ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
42622025-06-03T00:54:40.506ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 8 reason = Timeout
42632025-06-03T00:54:40.506ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 8 reason = Timeout
42642025-06-03T00:54:40.506ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 8 reason = Timeout
42652025-06-03T00:54:40.506ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
42662025-06-03T00:54:40.506ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
42672025-06-03T00:54:40.506ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 8 reason = Timeout
42682025-06-03T00:54:40.506ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
42692025-06-03T00:54:40.506ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
42702025-06-03T00:54:40.506ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
42712025-06-03T00:54:40.506ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 8 reason = Timeout
42722025-06-03T00:54:40.506ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
42732025-06-03T00:54:40.506ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 9 reason = Timeout
42742025-06-03T00:54:40.506ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
42752025-06-03T00:54:40.506ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 8 reason = Timeout
42762025-06-03T00:54:40.506ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 22 reason = Timeout
42772025-06-03T00:54:40.506ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
42782025-06-03T00:54:40.506ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 8 reason = Timeout
42792025-06-03T00:54:40.506ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
42802025-06-03T00:54:40.506ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
42812025-06-03T00:54:40.506ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 8 reason = Timeout
42822025-06-03T00:54:40.506ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
42832025-06-03T00:54:40.506ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 8 reason = Timeout
42842025-06-03T00:54:40.506ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
42852025-06-03T00:54:40.506ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 8 reason = Timeout
42862025-06-03T00:54:40.506ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
42872025-06-03T00:54:40.506ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 8 reason = Timeout
42882025-06-03T00:54:40.506ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
42892025-06-03T00:54:40.506ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 8 reason = Timeout
42902025-06-03T00:54:40.506ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
42912025-06-03T00:54:40.506ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 8 reason = Timeout
42922025-06-03T00:54:40.506ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
42932025-06-03T00:54:40.506ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 8 reason = Timeout
42942025-06-03T00:54:40.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
42952025-06-03T00:54:40.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
42962025-06-03T00:54:40.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
42972025-06-03T00:54:40.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
42982025-06-03T00:54:40.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
42992025-06-03T00:54:40.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
43002025-06-03T00:54:40.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
43012025-06-03T00:54:40.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
43022025-06-03T00:54:40.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
43032025-06-03T00:54:40.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
43042025-06-03T00:54:40.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
43052025-06-03T00:54:40.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
43062025-06-03T00:54:40.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
43072025-06-03T00:54:40.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
43082025-06-03T00:54:40.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
43092025-06-03T00:54:40.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
43102025-06-03T00:54:40.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
43112025-06-03T00:54:40.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
43122025-06-03T00:54:40.509ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-afe89ed7-19b7-479b-81eb-05c988ebc66d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-afe89ed7-19b7-479b-81eb-05c988ebc66d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 2607, tv_nsec: 733130865 } })
43132025-06-03T00:54:40.509ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1179
43142025-06-03T00:54:40.509ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43152025-06-03T00:54:40.509ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43162025-06-03T00:54:40.510ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.732806ms iteration = 8 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
43172025-06-03T00:54:40.510ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43182025-06-03T00:54:40.511ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43192025-06-03T00:54:40.513ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
43202025-06-03T00:54:40.513ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
43212025-06-03T00:54:40.513ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
43222025-06-03T00:54:40.513ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
43232025-06-03T00:54:40.513ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
43242025-06-03T00:54:40.513ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
43252025-06-03T00:54:40.513ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
43262025-06-03T00:54:40.513ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
43272025-06-03T00:54:40.514ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
43282025-06-03T00:54:40.514ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
43292025-06-03T00:54:40.514ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
43302025-06-03T00:54:40.514ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
43312025-06-03T00:54:40.515ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 9.063732ms iteration = 8 status = {"errors":[],"finish_invoked_ok":[]}
43322025-06-03T00:54:40.516ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43332025-06-03T00:54:40.517ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
43342025-06-03T00:54:40.517ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
43352025-06-03T00:54:40.517ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
43362025-06-03T00:54:40.517ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
43372025-06-03T00:54:40.518ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
43382025-06-03T00:54:40.518ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
43392025-06-03T00:54:40.519ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
43402025-06-03T00:54:40.519ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
43412025-06-03T00:54:40.519ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
43422025-06-03T00:54:40.520ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43432025-06-03T00:54:40.522ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-afe89ed7-19b7-479b-81eb-05c988ebc66d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-afe89ed7-19b7-479b-81eb-05c988ebc66d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 2607, tv_nsec: 746154719 } })
43442025-06-03T00:54:40.522ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
43452025-06-03T00:54:40.522ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1179
43462025-06-03T00:54:40.522ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
43472025-06-03T00:54:40.522ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43482025-06-03T00:54:40.522ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43492025-06-03T00:54:40.522ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
43502025-06-03T00:54:40.522ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
43512025-06-03T00:54:40.523ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
43522025-06-03T00:54:40.523ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
43532025-06-03T00:54:40.523ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
43542025-06-03T00:54:40.524ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
43552025-06-03T00:54:40.524ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
43562025-06-03T00:54:40.525ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
43572025-06-03T00:54:40.525ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
43582025-06-03T00:54:40.525ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
43592025-06-03T00:54:40.525ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
43602025-06-03T00:54:40.525ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
43612025-06-03T00:54:40.525ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
43622025-06-03T00:54:40.525ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
43632025-06-03T00:54:40.526ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
43642025-06-03T00:54:40.527ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
43652025-06-03T00:54:40.527ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
43662025-06-03T00:54:40.527ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
43672025-06-03T00:54:40.527ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
43682025-06-03T00:54:40.527ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 21.2629ms iteration = 8 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
43692025-06-03T00:54:40.533ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43702025-06-03T00:54:40.533ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:547 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
43712025-06-03T00:54:40.533ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 26.781161ms iteration = 8 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
43722025-06-03T00:54:40.534ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-afe89ed7-19b7-479b-81eb-05c988ebc66d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-afe89ed7-19b7-479b-81eb-05c988ebc66d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 2607, tv_nsec: 758477292 } })
43732025-06-03T00:54:40.534ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1179 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae
43742025-06-03T00:54:40.535ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43752025-06-03T00:54:40.536ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 4ec93699-353f-4b9e-ae06-25355fdb069e, lookup_type: ById(4ec93699-353f-4b9e-ae06-25355fdb069e) } result = Ok(())
43762025-06-03T00:54:40.536ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
43772025-06-03T00:54:40.536ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
43782025-06-03T00:54:40.537ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
43792025-06-03T00:54:40.537ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
43802025-06-03T00:54:40.537ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
43812025-06-03T00:54:40.545ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
43822025-06-03T00:54:40.545ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
43832025-06-03T00:54:40.549ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
43842025-06-03T00:54:40.549ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
43852025-06-03T00:54:40.556ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 50.257231ms iteration = 8 status = {"errors":[],"garbage_collect_requested":[]}
43862025-06-03T00:54:40.558ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
43872025-06-03T00:54:40.559ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
43882025-06-03T00:54:40.576ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 69.942231ms iteration = 8 status = {"errors":[],"requests_created_ok":[]}
43892025-06-03T00:54:40.579ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
43902025-06-03T00:54:40.579ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
43912025-06-03T00:54:40.579ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
43922025-06-03T00:54:40.580ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
43932025-06-03T00:54:40.581ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 74.879055ms iteration = 8 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
43942025-06-03T00:54:40.583ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
43952025-06-03T00:54:40.583ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
43962025-06-03T00:54:40.587ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1187 zone_address = fd00:1122:3344:101::2
43972025-06-03T00:54:40.587ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
43982025-06-03T00:54:40.587ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "d25ed9d3-6f2e-425b-92a7-f9a5ef7733ee", "content-length": "26", "date": "Tue, 03 Jun 2025 00:54:40 GMT"} })
43992025-06-03T00:54:40.587ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1190 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
44002025-06-03T00:54:40.589ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1224 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
44012025-06-03T00:54:40.619ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1187 zone_address = fd00:1122:3344:101::2
44022025-06-03T00:54:40.619ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
44032025-06-03T00:54:40.619ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "78a69f5a-1c8b-4aee-b31b-c2dcfb879ce5", "content-length": "26", "date": "Tue, 03 Jun 2025 00:54:40 GMT"} })
44042025-06-03T00:54:40.619ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1190 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
44052025-06-03T00:54:40.620ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1224 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
44062025-06-03T00:54:40.646ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1187 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae zone_address = fd00:1122:3344:101::2
44072025-06-03T00:54:40.646ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
44082025-06-03T00:54:40.647ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "819a8e70-130d-407c-a6d2-a410cd36e788", "content-length": "26", "date": "Tue, 03 Jun 2025 00:54:40 GMT"} })
44092025-06-03T00:54:40.647ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1190 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
44102025-06-03T00:54:40.648ZINFOnexus (BackgroundTasks): VPC route manager sled afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
44112025-06-03T00:54:40.648ZDEBGnexus (BackgroundTasks): client request SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
44122025-06-03T00:54:40.649ZDEBGnexus (BackgroundTasks): client response SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "f74c57f8-525f-4b7c-b2d1-7d55b2d5370d", "content-length": "372", "date": "Tue, 03 Jun 2025 00:54:40 GMT"} })
44132025-06-03T00:54:40.649ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
44142025-06-03T00:54:40.649ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1224 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae
44152025-06-03T00:54:40.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
44162025-06-03T00:54:40.651ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
44172025-06-03T00:54:40.652ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
44182025-06-03T00:54:40.653ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
44192025-06-03T00:54:40.653ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
44202025-06-03T00:54:40.654ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
44212025-06-03T00:54:40.654ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
44222025-06-03T00:54:40.654ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
44232025-06-03T00:54:40.655ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
44242025-06-03T00:54:40.663ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42589 req_id = b3539dfe-dd8d-4d41-9cf6-91d52f0ad68e roles = RoleSet { roles: {} } uri = /metrics/collectors/292a19e1-4305-4e49-b11e-08ac74714bf3/producers?sort_by=id_ascending
44252025-06-03T00:54:40.663ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 4ec93699-353f-4b9e-ae06-25355fdb069e original_time_created = 2025-06-03 00:51:07.059326 UTC target_id = 4ec93699-353f-4b9e-ae06-25355fdb069e time_created = 2025-06-03 00:51:07.059326 UTC
44262025-06-03T00:54:40.663ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 156.973164ms iteration = 22 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"4ec93699-353f-4b9e-ae06-25355fdb069e","time_created":"2025-06-03T00:51:07.059326Z"}
44272025-06-03T00:54:40.663ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42589 req_id = b3539dfe-dd8d-4d41-9cf6-91d52f0ad68e resource = Database result = Ok(()) uri = /metrics/collectors/292a19e1-4305-4e49-b11e-08ac74714bf3/producers?sort_by=id_ascending
44282025-06-03T00:54:40.671ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 8723 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42589 req_id = b3539dfe-dd8d-4d41-9cf6-91d52f0ad68e response_code = 200 uri = /metrics/collectors/292a19e1-4305-4e49-b11e-08ac74714bf3/producers?sort_by=id_ascending
44292025-06-03T00:54:40.672ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42589 req_id = cde9ab43-1e94-42b0-89b9-e17b7e6b21c5 roles = RoleSet { roles: {} } uri = /metrics/collectors/292a19e1-4305-4e49-b11e-08ac74714bf3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmZTEyZmZjMS1mMmFjLTRiZjMtOTY3Mi1mYzE4NWY4MmM1OWIifX0%3D
44302025-06-03T00:54:40.672ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42589 req_id = cde9ab43-1e94-42b0-89b9-e17b7e6b21c5 resource = Database result = Ok(()) uri = /metrics/collectors/292a19e1-4305-4e49-b11e-08ac74714bf3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmZTEyZmZjMS1mMmFjLTRiZjMtOTY3Mi1mYzE4NWY4MmM1OWIifX0%3D
44312025-06-03T00:54:40.675ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 3365 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42589 req_id = cde9ab43-1e94-42b0-89b9-e17b7e6b21c5 response_code = 200 uri = /metrics/collectors/292a19e1-4305-4e49-b11e-08ac74714bf3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmZTEyZmZjMS1mMmFjLTRiZjMtOTY3Mi1mYzE4NWY4MmM1OWIifX0%3D
44322025-06-03T00:54:40.694ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 187.41517ms iteration = 8 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
44332025-06-03T00:54:40.696ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:372
44342025-06-03T00:54:40.696ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 190.528026ms iteration = 8 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
44352025-06-03T00:54:40.702ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 196.008996ms iteration = 22 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
44362025-06-03T00:54:40.707ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
44372025-06-03T00:54:40.708ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
44382025-06-03T00:54:40.708ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
44392025-06-03T00:54:40.708ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 202.388244ms iteration = 8 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
44402025-06-03T00:54:40.709ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
44412025-06-03T00:54:40.709ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
44422025-06-03T00:54:40.709ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
44432025-06-03T00:54:40.712ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
44442025-06-03T00:54:40.712ZDEBGnexus (BackgroundTasks): client request SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
44452025-06-03T00:54:40.712ZDEBGnexus (BackgroundTasks): client response SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "f6beaf13-f8a0-49bc-99b6-6c0aae3968d6", "date": "Tue, 03 Jun 2025 00:54:40 GMT"} })
44462025-06-03T00:54:40.712ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
44472025-06-03T00:54:40.712ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
44482025-06-03T00:54:40.713ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 207.350929ms iteration = 9 status = {"physical_disks_added":0}
44492025-06-03T00:54:40.715ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
44502025-06-03T00:54:40.715ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
44512025-06-03T00:54:40.717ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
44522025-06-03T00:54:40.718ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
44532025-06-03T00:54:40.777ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
44542025-06-03T00:54:40.777ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
44552025-06-03T00:54:40.800ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 294.213946ms iteration = 8 status = {}
44562025-06-03T00:54:40.805ZDEBGnexus (BackgroundTasks): client request SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
44572025-06-03T00:54:40.808ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 301.597686ms iteration = 8 status = null
44582025-06-03T00:54:40.809ZINFOnexus (BackgroundTasks): VPC route manager sled afe89ed7-19b7-479b-81eb-05c988ebc66d push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
44592025-06-03T00:54:40.809ZDEBGnexus (BackgroundTasks): client request SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
44602025-06-03T00:54:40.809ZDEBGnexus (BackgroundTasks): client response SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "6768a66c-4f54-47df-9ab6-c78700f983fb", "content-length": "691", "date": "Tue, 03 Jun 2025 00:54:40 GMT"} })
44612025-06-03T00:54:40.809ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
44622025-06-03T00:54:40.809ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
44632025-06-03T00:54:40.809ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
44642025-06-03T00:54:40.811ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 304.719912ms iteration = 8 status = {}
44652025-06-03T00:54:40.813ZDEBGnexus (BackgroundTasks): client response SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "763fda6f-4a84-41f4-8535-ae6827aa9fd2", "date": "Tue, 03 Jun 2025 00:54:40 GMT"} })
44662025-06-03T00:54:40.815ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 308.787709ms iteration = 8 status = {}
44672025-06-03T00:54:40.848ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
44682025-06-03T00:54:40.848ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
44692025-06-03T00:54:40.850ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: ec5e33f9-c697-46e2-a93e-a745336bf9dc, rack_id: d6831ba0-55f0-4191-8dc2-ea15e55ab0ae, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(9378a284-0ee9-48e2-80a0-3bcaadb1c4e2) } rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae switch_location = Switch0
44702025-06-03T00:54:40.850ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
44712025-06-03T00:54:40.851ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
44722025-06-03T00:54:40.862ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1832 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
44732025-06-03T00:54:40.862ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1859 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 9378a284-0ee9-48e2-80a0-3bcaadb1c4e2
44742025-06-03T00:54:40.862ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
44752025-06-03T00:54:40.862ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
44762025-06-03T00:54:40.862ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
44772025-06-03T00:54:40.862ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
44782025-06-03T00:54:40.862ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
44792025-06-03T00:54:40.862ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2006 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
44802025-06-03T00:54:40.862ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:408 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
44812025-06-03T00:54:40.862ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2042 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
44822025-06-03T00:54:40.862ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae
44832025-06-03T00:54:40.862ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
44842025-06-03T00:54:40.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
44852025-06-03T00:54:40.865ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae switch_location = Switch0
44862025-06-03T00:54:40.865ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
44872025-06-03T00:54:40.874ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
44882025-06-03T00:54:40.874ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
44892025-06-03T00:54:40.876ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
44902025-06-03T00:54:40.876ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
44912025-06-03T00:54:40.878ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
44922025-06-03T00:54:40.880ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: afe89ed7-19b7-479b-81eb-05c988ebc66d, lookup_type: ById(afe89ed7-19b7-479b-81eb-05c988ebc66d) } result = Ok(())
44932025-06-03T00:54:40.880ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
44942025-06-03T00:54:40.881ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
44952025-06-03T00:54:40.882ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 376.091748ms iteration = 8 status = 0
44962025-06-03T00:54:40.999ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "b914c1b0-49ad-4c81-975e-a614416e8ffa", "date": "Tue, 03 Jun 2025 00:54:40 GMT"} })
44972025-06-03T00:54:40.999ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae switch = Switch0
44982025-06-03T00:54:40.999ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
44992025-06-03T00:54:40.999ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "0a712d6b-b1f0-4bc1-b4e0-8e1b3e73faca", "content-length": "682", "date": "Tue, 03 Jun 2025 00:54:40 GMT"} })
45002025-06-03T00:54:40.999ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
45012025-06-03T00:54:40.999ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
45022025-06-03T00:54:41.003ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
45032025-06-03T00:54:41.003ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
45042025-06-03T00:54:41.006ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
45052025-06-03T00:54:41.007ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
45062025-06-03T00:54:41.009ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
45072025-06-03T00:54:41.009ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
45082025-06-03T00:54:41.016ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 509.46242ms iteration = 8 status = {}
45092025-06-03T00:54:41.077ZDEBGnexus (producer-registration-task): registering as oximeter producer with Nexus registration_count = 1
45102025-06-03T00:54:41.077ZDEBGnexus (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::a]:12221
45112025-06-03T00:54:41.107ZDEBGnexus (producer-registration-task): client request body = Some(Body) method = POST uri = http://[fd00:1122:3344:101::a]:12221/metrics/producers
45122025-06-03T00:54:41.107ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:1025 local_addr = [fd00:1122:3344:101::a]:12221 remote_addr = [fd00:1122:3344:101::a]:56412
45132025-06-03T00:54:41.107ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = POST remote_addr = [fd00:1122:3344:101::a]:56412 req_id = fbc06cf7-d896-4fe8-9403-c4ab4206c0bf roles = RoleSet { roles: {} } uri = /metrics/producers
45142025-06-03T00:54:41.107ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = POST remote_addr = [fd00:1122:3344:101::a]:56412 req_id = fbc06cf7-d896-4fe8-9403-c4ab4206c0bf resource = Database result = Ok(()) uri = /metrics/producers
45152025-06-03T00:54:41.153ZINFOnexus: assigned collector to new producer collector_id = 292a19e1-4305-4e49-b11e-08ac74714bf3 file = nexus/src/app/oximeter.rs:76 producer_id = a2861392-5fd2-462b-84e5-6f36c716263a
45162025-06-03T00:54:41.153ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 46612 local_addr = [fd00:1122:3344:101::a]:12221 method = POST remote_addr = [fd00:1122:3344:101::a]:56412 req_id = fbc06cf7-d896-4fe8-9403-c4ab4206c0bf response_code = 201 uri = /metrics/producers
45172025-06-03T00:54:41.153ZDEBGnexus (producer-registration-task): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::a]:12221/metrics/producers", status: 201, headers: {"content-type": "application/json", "x-request-id": "fbc06cf7-d896-4fe8-9403-c4ab4206c0bf", "content-length": "41", "date": "Tue, 03 Jun 2025 00:54:41 GMT"} })
45182025-06-03T00:54:41.156ZDEBGnexus (producer-registration-task): successfully registered with Nexus, pausing until time to renew lease lease_duration = 600s wait_period = 150s
45192025-06-03T00:54:45.673ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 454 local_addr = [fd00:1122:3344:101::a]:63576 method = GET remote_addr = [fd00:1122:3344:101::d]:61315 req_id = d46b1574-9ef1-4f77-b908-fc2709353c89 response_code = 200 uri = /a2861392-5fd2-462b-84e5-6f36c716263a
45202025-06-03T00:54:50.505ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 23 reason = Timeout
45212025-06-03T00:54:50.505ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 23 reason = Timeout
45222025-06-03T00:54:50.505ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
45232025-06-03T00:54:50.505ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
45242025-06-03T00:54:50.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
45252025-06-03T00:54:50.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
45262025-06-03T00:54:50.507ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45272025-06-03T00:54:50.510ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
45282025-06-03T00:54:50.510ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
45292025-06-03T00:54:50.510ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
45302025-06-03T00:54:50.512ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
45312025-06-03T00:54:50.512ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
45322025-06-03T00:54:50.514ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45332025-06-03T00:54:50.516ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 4ec93699-353f-4b9e-ae06-25355fdb069e, lookup_type: ById(4ec93699-353f-4b9e-ae06-25355fdb069e) } result = Ok(())
45342025-06-03T00:54:50.516ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
45352025-06-03T00:54:50.516ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
45362025-06-03T00:54:50.528ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 4ec93699-353f-4b9e-ae06-25355fdb069e original_time_created = 2025-06-03 00:51:07.059326 UTC target_id = 4ec93699-353f-4b9e-ae06-25355fdb069e time_created = 2025-06-03 00:51:07.059326 UTC
45372025-06-03T00:54:50.528ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 22.429456ms iteration = 23 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"4ec93699-353f-4b9e-ae06-25355fdb069e","time_created":"2025-06-03T00:51:07.059326Z"}
45382025-06-03T00:54:50.538ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
45392025-06-03T00:54:50.538ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
45402025-06-03T00:54:50.569ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 63.953486ms iteration = 23 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
45412025-06-03T00:54:55.662ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42589 req_id = 96937523-e791-47c4-a58c-e1c1c493e86c roles = RoleSet { roles: {} } uri = /metrics/collectors/292a19e1-4305-4e49-b11e-08ac74714bf3/producers?sort_by=id_ascending
45422025-06-03T00:54:55.662ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42589 req_id = 96937523-e791-47c4-a58c-e1c1c493e86c resource = Database result = Ok(()) uri = /metrics/collectors/292a19e1-4305-4e49-b11e-08ac74714bf3/producers?sort_by=id_ascending
45432025-06-03T00:54:55.665ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2675 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42589 req_id = 96937523-e791-47c4-a58c-e1c1c493e86c response_code = 200 uri = /metrics/collectors/292a19e1-4305-4e49-b11e-08ac74714bf3/producers?sort_by=id_ascending
45442025-06-03T00:54:55.665ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42589 req_id = fcb47320-6833-498c-b515-686bd00224f6 roles = RoleSet { roles: {} } uri = /metrics/collectors/292a19e1-4305-4e49-b11e-08ac74714bf3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmZTEyZmZjMS1mMmFjLTRiZjMtOTY3Mi1mYzE4NWY4MmM1OWIifX0%3D
45452025-06-03T00:54:55.665ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42589 req_id = fcb47320-6833-498c-b515-686bd00224f6 resource = Database result = Ok(()) uri = /metrics/collectors/292a19e1-4305-4e49-b11e-08ac74714bf3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmZTEyZmZjMS1mMmFjLTRiZjMtOTY3Mi1mYzE4NWY4MmM1OWIifX0%3D
45462025-06-03T00:54:55.667ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2155 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42589 req_id = fcb47320-6833-498c-b515-686bd00224f6 response_code = 200 uri = /metrics/collectors/292a19e1-4305-4e49-b11e-08ac74714bf3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmZTEyZmZjMS1mMmFjLTRiZjMtOTY3Mi1mYzE4NWY4MmM1OWIifX0%3D
45472025-06-03T00:54:55.673ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 281 local_addr = [fd00:1122:3344:101::a]:63576 method = GET remote_addr = [fd00:1122:3344:101::d]:61315 req_id = 37d334c0-c17b-447a-8d26-0a95b9e638ab response_code = 200 uri = /a2861392-5fd2-462b-84e5-6f36c716263a
45482025-06-03T00:55:00.505ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 24 reason = Timeout
45492025-06-03T00:55:00.505ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
45502025-06-03T00:55:00.505ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 24 reason = Timeout
45512025-06-03T00:55:00.505ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
45522025-06-03T00:55:00.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
45532025-06-03T00:55:00.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
45542025-06-03T00:55:00.507ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
45552025-06-03T00:55:00.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
45562025-06-03T00:55:00.508ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45572025-06-03T00:55:00.509ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.854669ms iteration = 24 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
45582025-06-03T00:55:00.510ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
45592025-06-03T00:55:00.511ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
45602025-06-03T00:55:00.511ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
45612025-06-03T00:55:00.513ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
45622025-06-03T00:55:00.513ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
45632025-06-03T00:55:00.514ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45642025-06-03T00:55:00.516ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 4ec93699-353f-4b9e-ae06-25355fdb069e, lookup_type: ById(4ec93699-353f-4b9e-ae06-25355fdb069e) } result = Ok(())
45652025-06-03T00:55:00.516ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
45662025-06-03T00:55:00.517ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
45672025-06-03T00:55:00.525ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 4ec93699-353f-4b9e-ae06-25355fdb069e original_time_created = 2025-06-03 00:51:07.059326 UTC target_id = 4ec93699-353f-4b9e-ae06-25355fdb069e time_created = 2025-06-03 00:51:07.059326 UTC
45682025-06-03T00:55:00.525ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 19.925238ms iteration = 24 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"4ec93699-353f-4b9e-ae06-25355fdb069e","time_created":"2025-06-03T00:51:07.059326Z"}
45692025-06-03T00:55:05.673ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 413 local_addr = [fd00:1122:3344:101::a]:63576 method = GET remote_addr = [fd00:1122:3344:101::d]:61315 req_id = e637ef60-ecc0-4104-b329-6b0da1ca790c response_code = 200 uri = /a2861392-5fd2-462b-84e5-6f36c716263a
45702025-06-03T00:55:10.505ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 5 reason = Timeout
45712025-06-03T00:55:10.505ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
45722025-06-03T00:55:10.505ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
45732025-06-03T00:55:10.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
45742025-06-03T00:55:10.505ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 25 reason = Timeout
45752025-06-03T00:55:10.505ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
45762025-06-03T00:55:10.505ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 9 reason = Timeout
45772025-06-03T00:55:10.505ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
45782025-06-03T00:55:10.505ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 5 reason = Timeout
45792025-06-03T00:55:10.505ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:53
45802025-06-03T00:55:10.505ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 5 reason = Timeout
45812025-06-03T00:55:10.505ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-06-03 00:45:10.505779658 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
45822025-06-03T00:55:10.505ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
45832025-06-03T00:55:10.505ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
45842025-06-03T00:55:10.505ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 5 reason = Timeout
45852025-06-03T00:55:10.505ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 5 reason = Timeout
45862025-06-03T00:55:10.505ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
45872025-06-03T00:55:10.505ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 5 reason = Timeout
45882025-06-03T00:55:10.505ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 5 reason = Timeout
45892025-06-03T00:55:10.505ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
45902025-06-03T00:55:10.505ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 5 reason = Timeout
45912025-06-03T00:55:10.505ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
45922025-06-03T00:55:10.505ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 7 reason = Timeout
45932025-06-03T00:55:10.505ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 5 reason = Timeout
45942025-06-03T00:55:10.505ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
45952025-06-03T00:55:10.505ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
45962025-06-03T00:55:10.505ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 9 reason = Timeout
45972025-06-03T00:55:10.505ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
45982025-06-03T00:55:10.505ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
45992025-06-03T00:55:10.505ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 5 reason = Timeout
46002025-06-03T00:55:10.505ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 6 reason = Timeout
46012025-06-03T00:55:10.506ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:85 reason = blueprint disabled target_id = 4ec93699-353f-4b9e-ae06-25355fdb069e
46022025-06-03T00:55:10.506ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
46032025-06-03T00:55:10.506ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 9 reason = Timeout
46042025-06-03T00:55:10.506ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 12.39\u{b5}s iteration = 6 status = {"enabled":false,"target_id":"4ec93699-353f-4b9e-ae06-25355fdb069e"}
46052025-06-03T00:55:10.506ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
46062025-06-03T00:55:10.506ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 7 reason = Timeout
46072025-06-03T00:55:10.506ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 9 reason = Timeout
46082025-06-03T00:55:10.506ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 9 reason = Timeout
46092025-06-03T00:55:10.506ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
46102025-06-03T00:55:10.506ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 9 reason = Timeout
46112025-06-03T00:55:10.506ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
46122025-06-03T00:55:10.506ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 9 reason = Timeout
46132025-06-03T00:55:10.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
46142025-06-03T00:55:10.506ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
46152025-06-03T00:55:10.506ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
46162025-06-03T00:55:10.506ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 10 reason = Timeout
46172025-06-03T00:55:10.506ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
46182025-06-03T00:55:10.506ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 9 reason = Timeout
46192025-06-03T00:55:10.506ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 9 reason = Timeout
46202025-06-03T00:55:10.506ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
46212025-06-03T00:55:10.506ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 9 reason = Timeout
46222025-06-03T00:55:10.506ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
46232025-06-03T00:55:10.506ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
46242025-06-03T00:55:10.506ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 9 reason = Timeout
46252025-06-03T00:55:10.506ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
46262025-06-03T00:55:10.506ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 25 reason = Timeout
46272025-06-03T00:55:10.506ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
46282025-06-03T00:55:10.506ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 9 reason = Timeout
46292025-06-03T00:55:10.506ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
46302025-06-03T00:55:10.506ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 9 reason = Timeout
46312025-06-03T00:55:10.506ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
46322025-06-03T00:55:10.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
46332025-06-03T00:55:10.506ZDEBGnexus (BackgroundTasks): activating background_task = webhook_deliverator iteration = 5 reason = Timeout
46342025-06-03T00:55:10.506ZDEBGnexus (BackgroundTasks): roles background_task = webhook_deliverator roles = RoleSet { roles: {} }
46352025-06-03T00:55:10.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
46362025-06-03T00:55:10.506ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 9 reason = Timeout
46372025-06-03T00:55:10.506ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 9 reason = Timeout
46382025-06-03T00:55:10.506ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 9 reason = Timeout
46392025-06-03T00:55:10.506ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
46402025-06-03T00:55:10.506ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
46412025-06-03T00:55:10.506ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
46422025-06-03T00:55:10.506ZDEBGnexus (BackgroundTasks): activating background_task = alert_dispatcher iteration = 5 reason = Timeout
46432025-06-03T00:55:10.506ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
46442025-06-03T00:55:10.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
46452025-06-03T00:55:10.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
46462025-06-03T00:55:10.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
46472025-06-03T00:55:10.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
46482025-06-03T00:55:10.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
46492025-06-03T00:55:10.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
46502025-06-03T00:55:10.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
46512025-06-03T00:55:10.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
46522025-06-03T00:55:10.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
46532025-06-03T00:55:10.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
46542025-06-03T00:55:10.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
46552025-06-03T00:55:10.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
46562025-06-03T00:55:10.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
46572025-06-03T00:55:10.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
46582025-06-03T00:55:10.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
46592025-06-03T00:55:10.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
46602025-06-03T00:55:10.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
46612025-06-03T00:55:10.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
46622025-06-03T00:55:10.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
46632025-06-03T00:55:10.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = webhook_deliverator resource = Database result = Ok(())
46642025-06-03T00:55:10.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
46652025-06-03T00:55:10.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
46662025-06-03T00:55:10.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
46672025-06-03T00:55:10.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = alert_dispatcher resource = Database result = Ok(())
46682025-06-03T00:55:10.507ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("9feb16c9-ff60-43e8-917b-965124f46e0d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("aa124775-31de-4648-981a-4af500268b66.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("aa124775-31de-4648-981a-4af500268b66.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::9))) }], valid_until: Instant { tv_sec: 2637, tv_nsec: 731390280 } })
46692025-06-03T00:55:10.507ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("241573bc-467c-4802-9f4b-f7c82da4e12e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("64fc0a69-d4b8-4f6c-9a07-77a4cfa94962.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("ceb74144-4b6b-4b3f-bc80-eac2d75f3bc9.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("ceb74144-4b6b-4b3f-bc80-eac2d75f3bc9.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:2::1))) }], valid_until: Instant { tv_sec: 2637, tv_nsec: 731550776 } })
46702025-06-03T00:55:10.507ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
46712025-06-03T00:55:10.507ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.789556ms iteration = 5 status = {"addresses":["[fd00:1122:3344:101::8]:5353","[fd00:1122:3344:101::9]:5353"]}
46722025-06-03T00:55:10.507ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } dns_group = internal
46732025-06-03T00:55:10.507ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.869369ms iteration = 5 status = {"addresses":["[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:3::1]:5353","[fd00:1122:3344:2::1]:5353"]}
46742025-06-03T00:55:10.512ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
46752025-06-03T00:55:10.512ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
46762025-06-03T00:55:10.514ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
46772025-06-03T00:55:10.517ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-06-03 00:45:10.505779658 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
46782025-06-03T00:55:10.517ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 11.246059ms iteration = 5 status = {"expiration":"2025-06-03T00:45:10.505779658Z","pruned":[]}
46792025-06-03T00:55:10.518ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
46802025-06-03T00:55:10.518ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
46812025-06-03T00:55:10.518ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
46822025-06-03T00:55:10.518ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 12.8864ms iteration = 9 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
46832025-06-03T00:55:10.525ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 20.014921ms iteration = 5 status = {"errors":[],"found_port_ok":[]}
46842025-06-03T00:55:10.526ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
46852025-06-03T00:55:10.528ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
46862025-06-03T00:55:10.528ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
46872025-06-03T00:55:10.529ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
46882025-06-03T00:55:10.529ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:167 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
46892025-06-03T00:55:10.529ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 23.626194ms iteration = 5 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
46902025-06-03T00:55:10.529ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
46912025-06-03T00:55:10.529ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
46922025-06-03T00:55:10.530ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
46932025-06-03T00:55:10.530ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
46942025-06-03T00:55:10.531ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
46952025-06-03T00:55:10.533ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
46962025-06-03T00:55:10.533ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
46972025-06-03T00:55:10.534ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
46982025-06-03T00:55:10.534ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
46992025-06-03T00:55:10.534ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
47002025-06-03T00:55:10.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
47012025-06-03T00:55:10.534ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
47022025-06-03T00:55:10.534ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
47032025-06-03T00:55:10.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
47042025-06-03T00:55:10.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
47052025-06-03T00:55:10.539ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47062025-06-03T00:55:10.542ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
47072025-06-03T00:55:10.542ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
47082025-06-03T00:55:10.542ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
47092025-06-03T00:55:10.550ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-afe89ed7-19b7-479b-81eb-05c988ebc66d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-afe89ed7-19b7-479b-81eb-05c988ebc66d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 2637, tv_nsec: 774357985 } })
47102025-06-03T00:55:10.550ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1179
47112025-06-03T00:55:10.553ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47122025-06-03T00:55:10.555ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-afe89ed7-19b7-479b-81eb-05c988ebc66d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-afe89ed7-19b7-479b-81eb-05c988ebc66d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 2637, tv_nsec: 779988400 } })
47132025-06-03T00:55:10.556ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1179
47142025-06-03T00:55:10.556ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
47152025-06-03T00:55:10.556ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
47162025-06-03T00:55:10.557ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
47172025-06-03T00:55:10.563ZINFOnexus (BackgroundTasks): all webhook delivery tasks completed background_task = webhook_deliverator file = nexus/src/app/background/tasks/webhook_deliverator.rs:202 num_receivers = 0
47182025-06-03T00:55:10.564ZDEBGnexus (BackgroundTasks): activation complete background_task = webhook_deliverator elapsed = 57.682161ms iteration = 5 status = {"by_rx":{},"error":null}
47192025-06-03T00:55:10.565ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47202025-06-03T00:55:10.568ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
47212025-06-03T00:55:10.568ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
47222025-06-03T00:55:10.568ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
47232025-06-03T00:55:10.570ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
47242025-06-03T00:55:10.570ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
47252025-06-03T00:55:10.570ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47262025-06-03T00:55:10.572ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
47272025-06-03T00:55:10.572ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
47282025-06-03T00:55:10.573ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
47292025-06-03T00:55:10.574ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
47302025-06-03T00:55:10.575ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
47312025-06-03T00:55:10.577ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47322025-06-03T00:55:10.580ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
47332025-06-03T00:55:10.580ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
47342025-06-03T00:55:10.580ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
47352025-06-03T00:55:10.581ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
47362025-06-03T00:55:10.586ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
47372025-06-03T00:55:10.586ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
47382025-06-03T00:55:10.586ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 81.15823ms iteration = 9 status = {"errors":[],"finish_invoked_ok":[]}
47392025-06-03T00:55:10.587ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
47402025-06-03T00:55:10.587ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = alert_dispatcher resource = Database result = Ok(())
47412025-06-03T00:55:10.589ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47422025-06-03T00:55:10.590ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47432025-06-03T00:55:10.590ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
47442025-06-03T00:55:10.592ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
47452025-06-03T00:55:10.593ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
47462025-06-03T00:55:10.593ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
47472025-06-03T00:55:10.594ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 4ec93699-353f-4b9e-ae06-25355fdb069e, lookup_type: ById(4ec93699-353f-4b9e-ae06-25355fdb069e) } result = Ok(())
47482025-06-03T00:55:10.594ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
47492025-06-03T00:55:10.594ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
47502025-06-03T00:55:10.611ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47512025-06-03T00:55:10.613ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 107.544489ms iteration = 9 status = {"errors":[],"requests_created_ok":[]}
47522025-06-03T00:55:10.616ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
47532025-06-03T00:55:10.616ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
47542025-06-03T00:55:10.616ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
47552025-06-03T00:55:10.620ZDEBGnexus (BackgroundTasks): activation complete background_task = alert_dispatcher elapsed = 113.740961ms iteration = 5 status = {"dispatched":[],"errors":[],"glob_version":"144.0.0","globs_reprocessed":{},"no_receivers":[]}
47562025-06-03T00:55:10.622ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 116.589ms iteration = 9 status = {"errors":[],"garbage_collect_requested":[]}
47572025-06-03T00:55:10.626ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:547 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
47582025-06-03T00:55:10.627ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 120.986637ms iteration = 9 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
47592025-06-03T00:55:10.630ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47602025-06-03T00:55:10.631ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
47612025-06-03T00:55:10.632ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
47622025-06-03T00:55:10.633ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
47632025-06-03T00:55:10.633ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
47642025-06-03T00:55:10.633ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
47652025-06-03T00:55:10.633ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
47662025-06-03T00:55:10.633ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
47672025-06-03T00:55:10.636ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47682025-06-03T00:55:10.636ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 130.361718ms iteration = 25 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
47692025-06-03T00:55:10.637ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
47702025-06-03T00:55:10.637ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
47712025-06-03T00:55:10.638ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
47722025-06-03T00:55:10.638ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
47732025-06-03T00:55:10.638ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
47742025-06-03T00:55:10.639ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 133.894118ms iteration = 9 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
47752025-06-03T00:55:10.643ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-afe89ed7-19b7-479b-81eb-05c988ebc66d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-afe89ed7-19b7-479b-81eb-05c988ebc66d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 2637, tv_nsec: 868155177 } })
47762025-06-03T00:55:10.644ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1179 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae
47772025-06-03T00:55:10.646ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47782025-06-03T00:55:10.647ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
47792025-06-03T00:55:10.647ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:137 found = 0 not_ready_to_be_deleted = 0
47802025-06-03T00:55:10.647ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 141.457942ms iteration = 5 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
47812025-06-03T00:55:10.647ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
47822025-06-03T00:55:10.647ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
47832025-06-03T00:55:10.648ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
47842025-06-03T00:55:10.648ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 4ec93699-353f-4b9e-ae06-25355fdb069e original_time_created = 2025-06-03 00:51:07.059326 UTC target_id = 4ec93699-353f-4b9e-ae06-25355fdb069e time_created = 2025-06-03 00:51:07.059326 UTC
47852025-06-03T00:55:10.648ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 142.602368ms iteration = 25 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"4ec93699-353f-4b9e-ae06-25355fdb069e","time_created":"2025-06-03T00:51:07.059326Z"}
47862025-06-03T00:55:10.648ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
47872025-06-03T00:55:10.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
47882025-06-03T00:55:10.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
47892025-06-03T00:55:10.650ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 143.785275ms iteration = 9 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
47902025-06-03T00:55:10.650ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
47912025-06-03T00:55:10.652ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
47922025-06-03T00:55:10.652ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
47932025-06-03T00:55:10.652ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
47942025-06-03T00:55:10.652ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
47952025-06-03T00:55:10.652ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47962025-06-03T00:55:10.652ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
47972025-06-03T00:55:10.652ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
47982025-06-03T00:55:10.652ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
47992025-06-03T00:55:10.652ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
48002025-06-03T00:55:10.652ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
48012025-06-03T00:55:10.653ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
48022025-06-03T00:55:10.653ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
48032025-06-03T00:55:10.655ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
48042025-06-03T00:55:10.655ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
48052025-06-03T00:55:10.655ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
48062025-06-03T00:55:10.655ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
48072025-06-03T00:55:10.655ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
48082025-06-03T00:55:10.655ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
48092025-06-03T00:55:10.658ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
48102025-06-03T00:55:10.658ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
48112025-06-03T00:55:10.658ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
48122025-06-03T00:55:10.658ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
48132025-06-03T00:55:10.658ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
48142025-06-03T00:55:10.658ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
48152025-06-03T00:55:10.658ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
48162025-06-03T00:55:10.659ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
48172025-06-03T00:55:10.661ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
48182025-06-03T00:55:10.666ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
48192025-06-03T00:55:10.666ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
48202025-06-03T00:55:10.666ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
48212025-06-03T00:55:10.667ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1187 zone_address = fd00:1122:3344:101::2
48222025-06-03T00:55:10.667ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
48232025-06-03T00:55:10.667ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "134dc4dd-94ca-47c0-a630-d4e019af3266", "content-length": "26", "date": "Tue, 03 Jun 2025 00:55:10 GMT"} })
48242025-06-03T00:55:10.667ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1190 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
48252025-06-03T00:55:10.667ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
48262025-06-03T00:55:10.668ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
48272025-06-03T00:55:10.670ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1224 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
48282025-06-03T00:55:10.672ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
48292025-06-03T00:55:10.672ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
48302025-06-03T00:55:10.676ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
48312025-06-03T00:55:10.679ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: afe89ed7-19b7-479b-81eb-05c988ebc66d, lookup_type: ById(afe89ed7-19b7-479b-81eb-05c988ebc66d) } result = Ok(())
48322025-06-03T00:55:10.679ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
48332025-06-03T00:55:10.679ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
48342025-06-03T00:55:10.683ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 176.777849ms iteration = 9 status = 0
48352025-06-03T00:55:10.696ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1187 zone_address = fd00:1122:3344:101::2
48362025-06-03T00:55:10.696ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
48372025-06-03T00:55:10.696ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "805db07b-5f91-4902-aa10-c8dfbcedac4e", "content-length": "26", "date": "Tue, 03 Jun 2025 00:55:10 GMT"} })
48382025-06-03T00:55:10.696ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1190 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
48392025-06-03T00:55:10.698ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1224 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
48402025-06-03T00:55:10.706ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-06-03T00:51:07.585932Z, time_modified: 2025-06-03T00:51:07.585932Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: ea0ef520-6bf3-4fbd-9ff0-f3698b48b749, db_silo: Silo { identity: SiloIdentity { id: ea0ef520-6bf3-4fbd-9ff0-f3698b48b749, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-06-03T00:51:38.523369Z, time_modified: 2025-06-03T00:51:38.523369Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "bcda8a9f16e63c3cec49f9759478b9f335a661639158b0a9026f69b846d4e127" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: ea0ef520-6bf3-4fbd-9ff0-f3698b48b749, db_silo: Silo { identity: SiloIdentity { id: ea0ef520-6bf3-4fbd-9ff0-f3698b48b749, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-06-03T00:51:38.523369Z, time_modified: 2025-06-03T00:51:38.523369Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "bcda8a9f16e63c3cec49f9759478b9f335a661639158b0a9026f69b846d4e127" }] }) }
48412025-06-03T00:55:10.706ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 200.255158ms iteration = 5 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"ea0ef520-6bf3-4fbd-9ff0-f3698b48b749","tls_certs":["bcda8a9f16e63c3cec49f9759478b9f335a661639158b0a9026f69b846d4e127"]}},"default_endpoint":{"silo_id":"ea0ef520-6bf3-4fbd-9ff0-f3698b48b749","tls_certs":["bcda8a9f16e63c3cec49f9759478b9f335a661639158b0a9026f69b846d4e127"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
48422025-06-03T00:55:10.713ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:372
48432025-06-03T00:55:10.713ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 207.500913ms iteration = 9 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
48442025-06-03T00:55:10.713ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
48452025-06-03T00:55:10.714ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
48462025-06-03T00:55:10.718ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
48472025-06-03T00:55:10.718ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
48482025-06-03T00:55:10.719ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
48492025-06-03T00:55:10.719ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 9e79f9b1-a4de-4f9d-a901-16cc4f79e15d dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
48502025-06-03T00:55:10.719ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
48512025-06-03T00:55:10.719ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
48522025-06-03T00:55:10.719ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
48532025-06-03T00:55:10.719ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
48542025-06-03T00:55:10.719ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 213.885841ms iteration = 9 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
48552025-06-03T00:55:10.720ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 214.302894ms iteration = 9 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
48562025-06-03T00:55:10.721ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
48572025-06-03T00:55:10.724ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
48582025-06-03T00:55:10.724ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 3ef110b4-9400-4c37-8d7f-5552c8a94db0 dns_zone_name = oxide.test found_so_far = 0 version = 2
48592025-06-03T00:55:10.724ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
48602025-06-03T00:55:10.724ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
48612025-06-03T00:55:10.725ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
48622025-06-03T00:55:10.725ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
48632025-06-03T00:55:10.725ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
48642025-06-03T00:55:10.726ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
48652025-06-03T00:55:10.730ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
48662025-06-03T00:55:10.730ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
48672025-06-03T00:55:10.730ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 9e79f9b1-a4de-4f9d-a901-16cc4f79e15d dns_zone_name = control-plane.oxide.internal found_so_far = 59 version = 1
48682025-06-03T00:55:10.730ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
48692025-06-03T00:55:10.730ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2025-06-03 00:51:38.152773 UTC dns_group = internal generation = 1
48702025-06-03T00:55:10.730ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 225.702008ms iteration = 5 status = {"generation":1}
48712025-06-03T00:55:10.731ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
48722025-06-03T00:55:10.747ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
48732025-06-03T00:55:10.747ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
48742025-06-03T00:55:10.771ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1187 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae zone_address = fd00:1122:3344:101::2
48752025-06-03T00:55:10.771ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
48762025-06-03T00:55:10.771ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "cdd1b818-56c4-4ecf-a59a-42c3f9478d9b", "content-length": "26", "date": "Tue, 03 Jun 2025 00:55:10 GMT"} })
48772025-06-03T00:55:10.771ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1190 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
48782025-06-03T00:55:10.774ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1224 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae
48792025-06-03T00:55:10.784ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
48802025-06-03T00:55:10.787ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
48812025-06-03T00:55:10.787ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "13e211bd-1093-4bd4-a532-c7b3c38fbdb6", "date": "Tue, 03 Jun 2025 00:55:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
48822025-06-03T00:55:10.787ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
48832025-06-03T00:55:10.790ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "2772816e-2236-4583-9cef-66bdd4d8e4f6", "date": "Tue, 03 Jun 2025 00:55:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
48842025-06-03T00:55:10.790ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
48852025-06-03T00:55:10.791ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
48862025-06-03T00:55:10.791ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 285.822035ms iteration = 7 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
48872025-06-03T00:55:10.820ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 314.210527ms iteration = 10 status = {"physical_disks_added":0}
48882025-06-03T00:55:10.822ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 315.96006ms iteration = 9 status = {}
48892025-06-03T00:55:10.846ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
48902025-06-03T00:55:10.846ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
48912025-06-03T00:55:10.847ZINFOnexus (BackgroundTasks): VPC route manager sled afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
48922025-06-03T00:55:10.847ZDEBGnexus (BackgroundTasks): client request SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
48932025-06-03T00:55:10.847ZDEBGnexus (BackgroundTasks): client response SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "d6a72c02-ff44-4b00-b7de-9ce87482101a", "content-length": "372", "date": "Tue, 03 Jun 2025 00:55:10 GMT"} })
48942025-06-03T00:55:10.847ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
48952025-06-03T00:55:10.847ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
48962025-06-03T00:55:10.849ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 3ef110b4-9400-4c37-8d7f-5552c8a94db0 dns_zone_name = oxide.test found_so_far = 4 version = 2
48972025-06-03T00:55:10.849ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
48982025-06-03T00:55:10.849ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2025-06-03 00:51:38.567192 UTC dns_group = external generation = 2
48992025-06-03T00:55:10.849ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 343.360542ms iteration = 5 status = {"generation":2}
49002025-06-03T00:55:10.853ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 347.848171ms iteration = 9 status = null
49012025-06-03T00:55:10.882ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
49022025-06-03T00:55:10.882ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "5edc6b52-2a66-4b7c-b4e2-7e8eeffd177e", "date": "Tue, 03 Jun 2025 00:55:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
49032025-06-03T00:55:10.882ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
49042025-06-03T00:55:10.884ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "daaf4677-8552-40e3-9dce-fc1a445c8556", "date": "Tue, 03 Jun 2025 00:55:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
49052025-06-03T00:55:10.884ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
49062025-06-03T00:55:10.887ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "6af9af2c-653c-4e88-b850-e212fd471285", "date": "Tue, 03 Jun 2025 00:55:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
49072025-06-03T00:55:10.887ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
49082025-06-03T00:55:10.889ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
49092025-06-03T00:55:10.889ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 383.574091ms iteration = 7 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
49102025-06-03T00:55:10.890ZINFOnexus (BackgroundTasks): instance reincarnation completed successfully background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:113 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0
49112025-06-03T00:55:10.890ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 384.904263ms iteration = 5 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
49122025-06-03T00:55:10.899ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
49132025-06-03T00:55:10.899ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
49142025-06-03T00:55:10.931ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
49152025-06-03T00:55:10.931ZDEBGnexus (BackgroundTasks): client request SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
49162025-06-03T00:55:10.933ZDEBGnexus (BackgroundTasks): client response SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "f00ef2ba-a25a-4df6-a8db-9b87e3c8b216", "date": "Tue, 03 Jun 2025 00:55:10 GMT"} })
49172025-06-03T00:55:10.933ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
49182025-06-03T00:55:10.933ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
49192025-06-03T00:55:10.937ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
49202025-06-03T00:55:10.937ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
49212025-06-03T00:55:10.941ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
49222025-06-03T00:55:10.941ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
49232025-06-03T00:55:10.944ZINFOnexus (BackgroundTasks): VPC route manager sled afe89ed7-19b7-479b-81eb-05c988ebc66d push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
49242025-06-03T00:55:10.944ZDEBGnexus (BackgroundTasks): client request SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
49252025-06-03T00:55:10.945ZDEBGnexus (BackgroundTasks): client response SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "a0258b28-6867-425e-b691-93a6d0a4c903", "date": "Tue, 03 Jun 2025 00:55:10 GMT"} })
49262025-06-03T00:55:10.947ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 441.594933ms iteration = 9 status = {}
49272025-06-03T00:55:10.949ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
49282025-06-03T00:55:10.950ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
49292025-06-03T00:55:10.950ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
49302025-06-03T00:55:10.951ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
49312025-06-03T00:55:10.952ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
49322025-06-03T00:55:10.954ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
49332025-06-03T00:55:10.954ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
49342025-06-03T00:55:10.954ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
49352025-06-03T00:55:10.954ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
49362025-06-03T00:55:10.955ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
49372025-06-03T00:55:10.957ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: ec5e33f9-c697-46e2-a93e-a745336bf9dc, rack_id: d6831ba0-55f0-4191-8dc2-ea15e55ab0ae, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(9378a284-0ee9-48e2-80a0-3bcaadb1c4e2) } rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae switch_location = Switch0
49382025-06-03T00:55:10.957ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
49392025-06-03T00:55:10.957ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
49402025-06-03T00:55:11.070ZDEBGnexus (BackgroundTasks): client request SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
49412025-06-03T00:55:11.070ZDEBGnexus (BackgroundTasks): client response SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "b5d85ae0-b6d8-45a4-ad73-4e1f768a3b15", "content-length": "691", "date": "Tue, 03 Jun 2025 00:55:10 GMT"} })
49422025-06-03T00:55:11.070ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
49432025-06-03T00:55:11.070ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
49442025-06-03T00:55:11.070ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
49452025-06-03T00:55:11.071ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 565.653684ms iteration = 9 status = {}
49462025-06-03T00:55:11.079ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1832 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
49472025-06-03T00:55:11.080ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1859 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 9378a284-0ee9-48e2-80a0-3bcaadb1c4e2
49482025-06-03T00:55:11.080ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
49492025-06-03T00:55:11.080ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
49502025-06-03T00:55:11.080ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
49512025-06-03T00:55:11.080ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
49522025-06-03T00:55:11.080ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
49532025-06-03T00:55:11.080ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2006 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
49542025-06-03T00:55:11.080ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:408 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
49552025-06-03T00:55:11.080ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2042 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
49562025-06-03T00:55:11.080ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae
49572025-06-03T00:55:11.080ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
49582025-06-03T00:55:11.080ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
49592025-06-03T00:55:11.082ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae switch_location = Switch0
49602025-06-03T00:55:11.082ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
49612025-06-03T00:55:11.249ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "a4e915b4-d518-4b10-bff7-bb8197730fe4", "date": "Tue, 03 Jun 2025 00:55:10 GMT"} })
49622025-06-03T00:55:11.249ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae switch = Switch0
49632025-06-03T00:55:11.249ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
49642025-06-03T00:55:11.249ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "ad95b010-1754-435f-b8d7-f157168dd0ca", "content-length": "682", "date": "Tue, 03 Jun 2025 00:55:10 GMT"} })
49652025-06-03T00:55:11.249ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
49662025-06-03T00:55:11.250ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
49672025-06-03T00:55:11.256ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
49682025-06-03T00:55:11.256ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
49692025-06-03T00:55:11.260ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
49702025-06-03T00:55:11.260ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
49712025-06-03T00:55:11.262ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
49722025-06-03T00:55:11.262ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
49732025-06-03T00:55:11.269ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 763.28783ms iteration = 9 status = {}
49742025-06-03T00:55:15.674ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 453 local_addr = [fd00:1122:3344:101::a]:63576 method = GET remote_addr = [fd00:1122:3344:101::d]:61315 req_id = 5ffcf7d9-f966-4d9e-a5b8-ecb4d86cb3de response_code = 200 uri = /a2861392-5fd2-462b-84e5-6f36c716263a
49752025-06-03T00:55:20.505ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 26 reason = Timeout
49762025-06-03T00:55:20.505ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 26 reason = Timeout
49772025-06-03T00:55:20.505ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
49782025-06-03T00:55:20.505ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
49792025-06-03T00:55:20.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
49802025-06-03T00:55:20.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
49812025-06-03T00:55:20.507ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
49822025-06-03T00:55:20.509ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
49832025-06-03T00:55:20.509ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
49842025-06-03T00:55:20.509ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
49852025-06-03T00:55:20.511ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
49862025-06-03T00:55:20.511ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
49872025-06-03T00:55:20.513ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
49882025-06-03T00:55:20.515ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 4ec93699-353f-4b9e-ae06-25355fdb069e, lookup_type: ById(4ec93699-353f-4b9e-ae06-25355fdb069e) } result = Ok(())
49892025-06-03T00:55:20.515ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
49902025-06-03T00:55:20.515ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
49912025-06-03T00:55:20.524ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 4ec93699-353f-4b9e-ae06-25355fdb069e original_time_created = 2025-06-03 00:51:07.059326 UTC target_id = 4ec93699-353f-4b9e-ae06-25355fdb069e time_created = 2025-06-03 00:51:07.059326 UTC
49922025-06-03T00:55:20.524ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 19.088473ms iteration = 26 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"4ec93699-353f-4b9e-ae06-25355fdb069e","time_created":"2025-06-03T00:51:07.059326Z"}
49932025-06-03T00:55:20.538ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
49942025-06-03T00:55:20.538ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
49952025-06-03T00:55:20.571ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 66.530776ms iteration = 26 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
49962025-06-03T00:55:25.673ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 396 local_addr = [fd00:1122:3344:101::a]:63576 method = GET remote_addr = [fd00:1122:3344:101::d]:61315 req_id = dde35b98-7d7c-4832-bfe1-e592c5b686bd response_code = 200 uri = /a2861392-5fd2-462b-84e5-6f36c716263a
49972025-06-03T00:55:30.505ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 27 reason = Timeout
49982025-06-03T00:55:30.505ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
49992025-06-03T00:55:30.505ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 27 reason = Timeout
50002025-06-03T00:55:30.505ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
50012025-06-03T00:55:30.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
50022025-06-03T00:55:30.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
50032025-06-03T00:55:30.507ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
50042025-06-03T00:55:30.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
50052025-06-03T00:55:30.508ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
50062025-06-03T00:55:30.510ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
50072025-06-03T00:55:30.510ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
50082025-06-03T00:55:30.510ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
50092025-06-03T00:55:30.512ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
50102025-06-03T00:55:30.512ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
50112025-06-03T00:55:30.514ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
50122025-06-03T00:55:30.516ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 4ec93699-353f-4b9e-ae06-25355fdb069e, lookup_type: ById(4ec93699-353f-4b9e-ae06-25355fdb069e) } result = Ok(())
50132025-06-03T00:55:30.516ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
50142025-06-03T00:55:30.516ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
50152025-06-03T00:55:30.528ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 4ec93699-353f-4b9e-ae06-25355fdb069e original_time_created = 2025-06-03 00:51:07.059326 UTC target_id = 4ec93699-353f-4b9e-ae06-25355fdb069e time_created = 2025-06-03 00:51:07.059326 UTC
50162025-06-03T00:55:30.528ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 22.664043ms iteration = 27 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"4ec93699-353f-4b9e-ae06-25355fdb069e","time_created":"2025-06-03T00:51:07.059326Z"}
50172025-06-03T00:55:30.545ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 39.180967ms iteration = 27 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
50182025-06-03T00:55:35.673ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 285 local_addr = [fd00:1122:3344:101::a]:63576 method = GET remote_addr = [fd00:1122:3344:101::d]:61315 req_id = ce5d665e-218c-4ecc-8348-a00b8186ff3d response_code = 200 uri = /a2861392-5fd2-462b-84e5-6f36c716263a
50192025-06-03T00:55:40.505ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 28 reason = Timeout
50202025-06-03T00:55:40.505ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
50212025-06-03T00:55:40.505ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 10 reason = Timeout
50222025-06-03T00:55:40.505ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
50232025-06-03T00:55:40.505ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 10 reason = Timeout
50242025-06-03T00:55:40.505ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
50252025-06-03T00:55:40.505ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 10 reason = Timeout
50262025-06-03T00:55:40.505ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
50272025-06-03T00:55:40.505ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 10 reason = Timeout
50282025-06-03T00:55:40.505ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
50292025-06-03T00:55:40.505ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 10 reason = Timeout
50302025-06-03T00:55:40.505ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
50312025-06-03T00:55:40.505ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 10 reason = Timeout
50322025-06-03T00:55:40.505ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
50332025-06-03T00:55:40.505ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 10 reason = Timeout
50342025-06-03T00:55:40.505ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
50352025-06-03T00:55:40.505ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 10 reason = Timeout
50362025-06-03T00:55:40.505ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
50372025-06-03T00:55:40.505ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 10 reason = Timeout
50382025-06-03T00:55:40.505ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 10 reason = Timeout
50392025-06-03T00:55:40.505ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
50402025-06-03T00:55:40.505ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
50412025-06-03T00:55:40.505ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 10 reason = Timeout
50422025-06-03T00:55:40.505ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
50432025-06-03T00:55:40.505ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 10 reason = Timeout
50442025-06-03T00:55:40.505ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
50452025-06-03T00:55:40.505ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 10 reason = Timeout
50462025-06-03T00:55:40.505ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
50472025-06-03T00:55:40.505ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
50482025-06-03T00:55:40.505ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 10 reason = Timeout
50492025-06-03T00:55:40.505ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
50502025-06-03T00:55:40.505ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 11 reason = Timeout
50512025-06-03T00:55:40.505ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
50522025-06-03T00:55:40.505ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
50532025-06-03T00:55:40.505ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 10 reason = Timeout
50542025-06-03T00:55:40.505ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
50552025-06-03T00:55:40.505ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 10 reason = Timeout
50562025-06-03T00:55:40.505ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 28 reason = Timeout
50572025-06-03T00:55:40.505ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
50582025-06-03T00:55:40.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
50592025-06-03T00:55:40.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
50602025-06-03T00:55:40.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
50612025-06-03T00:55:40.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
50622025-06-03T00:55:40.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
50632025-06-03T00:55:40.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
50642025-06-03T00:55:40.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
50652025-06-03T00:55:40.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
50662025-06-03T00:55:40.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
50672025-06-03T00:55:40.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
50682025-06-03T00:55:40.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
50692025-06-03T00:55:40.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
50702025-06-03T00:55:40.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
50712025-06-03T00:55:40.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
50722025-06-03T00:55:40.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
50732025-06-03T00:55:40.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
50742025-06-03T00:55:40.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
50752025-06-03T00:55:40.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
50762025-06-03T00:55:40.508ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
50772025-06-03T00:55:40.509ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 3.771367ms iteration = 10 status = {"errors":[],"garbage_collect_requested":[]}
50782025-06-03T00:55:40.510ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
50792025-06-03T00:55:40.510ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
50802025-06-03T00:55:40.510ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
50812025-06-03T00:55:40.510ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 4.51489ms iteration = 10 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
50822025-06-03T00:55:40.510ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
50832025-06-03T00:55:40.510ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
50842025-06-03T00:55:40.510ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
50852025-06-03T00:55:40.511ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-afe89ed7-19b7-479b-81eb-05c988ebc66d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-afe89ed7-19b7-479b-81eb-05c988ebc66d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 2667, tv_nsec: 736099407 } })
50862025-06-03T00:55:40.511ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1179
50872025-06-03T00:55:40.512ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
50882025-06-03T00:55:40.512ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
50892025-06-03T00:55:40.512ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
50902025-06-03T00:55:40.512ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
50912025-06-03T00:55:40.515ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
50922025-06-03T00:55:40.516ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
50932025-06-03T00:55:40.516ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
50942025-06-03T00:55:40.516ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
50952025-06-03T00:55:40.516ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
50962025-06-03T00:55:40.518ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
50972025-06-03T00:55:40.519ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-afe89ed7-19b7-479b-81eb-05c988ebc66d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-afe89ed7-19b7-479b-81eb-05c988ebc66d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 2667, tv_nsec: 744464056 } })
50982025-06-03T00:55:40.520ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1179
50992025-06-03T00:55:40.521ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
51002025-06-03T00:55:40.521ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
51012025-06-03T00:55:40.521ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
51022025-06-03T00:55:40.521ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
51032025-06-03T00:55:40.521ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
51042025-06-03T00:55:40.525ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
51052025-06-03T00:55:40.525ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
51062025-06-03T00:55:40.525ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
51072025-06-03T00:55:40.525ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
51082025-06-03T00:55:40.525ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
51092025-06-03T00:55:40.525ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
51102025-06-03T00:55:40.529ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 23.697196ms iteration = 28 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
51112025-06-03T00:55:40.530ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
51122025-06-03T00:55:40.530ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
51132025-06-03T00:55:40.530ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
51142025-06-03T00:55:40.531ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
51152025-06-03T00:55:40.531ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
51162025-06-03T00:55:40.532ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
51172025-06-03T00:55:40.532ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
51182025-06-03T00:55:40.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
51192025-06-03T00:55:40.546ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
51202025-06-03T00:55:40.546ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
51212025-06-03T00:55:40.568ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1187 zone_address = fd00:1122:3344:101::2
51222025-06-03T00:55:40.568ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
51232025-06-03T00:55:40.569ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "6f7a7c06-b275-4628-b22a-3df24ccc5a66", "content-length": "26", "date": "Tue, 03 Jun 2025 00:55:40 GMT"} })
51242025-06-03T00:55:40.569ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1190 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
51252025-06-03T00:55:40.570ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 64.693449ms iteration = 10 status = {"errors":[],"finish_invoked_ok":[]}
51262025-06-03T00:55:40.570ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1224 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
51272025-06-03T00:55:40.573ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
51282025-06-03T00:55:40.573ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
51292025-06-03T00:55:40.573ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
51302025-06-03T00:55:40.573ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
51312025-06-03T00:55:40.576ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
51322025-06-03T00:55:40.577ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
51332025-06-03T00:55:40.578ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
51342025-06-03T00:55:40.578ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1187 zone_address = fd00:1122:3344:101::2
51352025-06-03T00:55:40.578ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
51362025-06-03T00:55:40.579ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
51372025-06-03T00:55:40.579ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "e8878666-e8e9-44d0-8bdb-3af99da699e9", "content-length": "26", "date": "Tue, 03 Jun 2025 00:55:40 GMT"} })
51382025-06-03T00:55:40.579ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1190 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
51392025-06-03T00:55:40.581ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1224 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
51402025-06-03T00:55:40.588ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 83.034359ms iteration = 10 status = {"errors":[],"requests_created_ok":[]}
51412025-06-03T00:55:40.588ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
51422025-06-03T00:55:40.589ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
51432025-06-03T00:55:40.590ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 84.552516ms iteration = 10 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
51442025-06-03T00:55:40.591ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 85.756863ms iteration = 10 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
51452025-06-03T00:55:40.591ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:547 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
51462025-06-03T00:55:40.591ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 86.336021ms iteration = 10 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
51472025-06-03T00:55:40.593ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
51482025-06-03T00:55:40.594ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
51492025-06-03T00:55:40.595ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
51502025-06-03T00:55:40.595ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
51512025-06-03T00:55:40.596ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
51522025-06-03T00:55:40.597ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
51532025-06-03T00:55:40.597ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
51542025-06-03T00:55:40.597ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
51552025-06-03T00:55:40.599ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
51562025-06-03T00:55:40.599ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
51572025-06-03T00:55:40.599ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
51582025-06-03T00:55:40.599ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
51592025-06-03T00:55:40.599ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
51602025-06-03T00:55:40.600ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
51612025-06-03T00:55:40.600ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
51622025-06-03T00:55:40.600ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
51632025-06-03T00:55:40.602ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: afe89ed7-19b7-479b-81eb-05c988ebc66d, lookup_type: ById(afe89ed7-19b7-479b-81eb-05c988ebc66d) } result = Ok(())
51642025-06-03T00:55:40.602ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
51652025-06-03T00:55:40.602ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
51662025-06-03T00:55:40.602ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
51672025-06-03T00:55:40.602ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
51682025-06-03T00:55:40.602ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
51692025-06-03T00:55:40.603ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-afe89ed7-19b7-479b-81eb-05c988ebc66d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-afe89ed7-19b7-479b-81eb-05c988ebc66d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 2667, tv_nsec: 827616218 } })
51702025-06-03T00:55:40.603ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1179 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae
51712025-06-03T00:55:40.605ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 99.975034ms iteration = 10 status = 0
51722025-06-03T00:55:40.612ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
51732025-06-03T00:55:40.617ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 4ec93699-353f-4b9e-ae06-25355fdb069e, lookup_type: ById(4ec93699-353f-4b9e-ae06-25355fdb069e) } result = Ok(())
51742025-06-03T00:55:40.617ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
51752025-06-03T00:55:40.617ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
51762025-06-03T00:55:40.631ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 125.311061ms iteration = 10 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
51772025-06-03T00:55:40.635ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
51782025-06-03T00:55:40.635ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
51792025-06-03T00:55:40.635ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
51802025-06-03T00:55:40.636ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
51812025-06-03T00:55:40.641ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 4ec93699-353f-4b9e-ae06-25355fdb069e original_time_created = 2025-06-03 00:51:07.059326 UTC target_id = 4ec93699-353f-4b9e-ae06-25355fdb069e time_created = 2025-06-03 00:51:07.059326 UTC
51822025-06-03T00:55:40.641ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 135.231319ms iteration = 28 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"4ec93699-353f-4b9e-ae06-25355fdb069e","time_created":"2025-06-03T00:51:07.059326Z"}
51832025-06-03T00:55:40.657ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
51842025-06-03T00:55:40.657ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
51852025-06-03T00:55:40.659ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:372
51862025-06-03T00:55:40.659ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 153.729973ms iteration = 10 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
51872025-06-03T00:55:40.659ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 153.816297ms iteration = 10 status = null
51882025-06-03T00:55:40.672ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
51892025-06-03T00:55:40.672ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
51902025-06-03T00:55:40.673ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 167.4794ms iteration = 10 status = {}
51912025-06-03T00:55:40.683ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
51922025-06-03T00:55:40.683ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
51932025-06-03T00:55:40.683ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 177.895054ms iteration = 10 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
51942025-06-03T00:55:40.684ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1187 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae zone_address = fd00:1122:3344:101::2
51952025-06-03T00:55:40.684ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
51962025-06-03T00:55:40.684ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "54de2d73-f258-43bc-9d9c-933655ca2648", "content-length": "26", "date": "Tue, 03 Jun 2025 00:55:40 GMT"} })
51972025-06-03T00:55:40.684ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1190 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
51982025-06-03T00:55:40.686ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1224 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae
51992025-06-03T00:55:40.719ZINFOnexus (BackgroundTasks): VPC route manager sled afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
52002025-06-03T00:55:40.719ZDEBGnexus (BackgroundTasks): client request SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
52012025-06-03T00:55:40.727ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 221.323653ms iteration = 11 status = {"physical_disks_added":0}
52022025-06-03T00:55:40.727ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
52032025-06-03T00:55:40.728ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
52042025-06-03T00:55:40.729ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
52052025-06-03T00:55:40.731ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
52062025-06-03T00:55:40.731ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
52072025-06-03T00:55:40.731ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
52082025-06-03T00:55:40.732ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
52092025-06-03T00:55:40.734ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
52102025-06-03T00:55:40.734ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
52112025-06-03T00:55:40.734ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
52122025-06-03T00:55:40.794ZDEBGnexus (BackgroundTasks): client response SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "637d872f-f6d3-4253-a6de-17e934f02dbf", "content-length": "372", "date": "Tue, 03 Jun 2025 00:55:40 GMT"} })
52132025-06-03T00:55:40.794ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
52142025-06-03T00:55:40.794ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
52152025-06-03T00:55:40.797ZDEBGnexus (BackgroundTasks): client request SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
52162025-06-03T00:55:40.799ZDEBGnexus (BackgroundTasks): client response SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "38f0f0ce-3b17-4b82-a4cd-3c69a5fc6512", "content-length": "691", "date": "Tue, 03 Jun 2025 00:55:40 GMT"} })
52172025-06-03T00:55:40.799ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
52182025-06-03T00:55:40.799ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
52192025-06-03T00:55:40.799ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
52202025-06-03T00:55:40.801ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 295.439854ms iteration = 10 status = {}
52212025-06-03T00:55:40.832ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
52222025-06-03T00:55:40.832ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
52232025-06-03T00:55:40.834ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: ec5e33f9-c697-46e2-a93e-a745336bf9dc, rack_id: d6831ba0-55f0-4191-8dc2-ea15e55ab0ae, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(9378a284-0ee9-48e2-80a0-3bcaadb1c4e2) } rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae switch_location = Switch0
52242025-06-03T00:55:40.834ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
52252025-06-03T00:55:40.835ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
52262025-06-03T00:55:40.843ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
52272025-06-03T00:55:40.843ZDEBGnexus (BackgroundTasks): client request SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
52282025-06-03T00:55:40.844ZDEBGnexus (BackgroundTasks): client response SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "ec41a83f-cd69-4f94-9153-1767692803ff", "date": "Tue, 03 Jun 2025 00:55:40 GMT"} })
52292025-06-03T00:55:40.844ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
52302025-06-03T00:55:40.844ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
52312025-06-03T00:55:40.846ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
52322025-06-03T00:55:40.846ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
52332025-06-03T00:55:40.847ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1832 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
52342025-06-03T00:55:40.847ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1859 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 9378a284-0ee9-48e2-80a0-3bcaadb1c4e2
52352025-06-03T00:55:40.847ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
52362025-06-03T00:55:40.847ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
52372025-06-03T00:55:40.847ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
52382025-06-03T00:55:40.847ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
52392025-06-03T00:55:40.847ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
52402025-06-03T00:55:40.847ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2006 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
52412025-06-03T00:55:40.848ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:408 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
52422025-06-03T00:55:40.848ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2042 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
52432025-06-03T00:55:40.848ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae
52442025-06-03T00:55:40.848ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
52452025-06-03T00:55:40.848ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
52462025-06-03T00:55:40.849ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
52472025-06-03T00:55:40.849ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
52482025-06-03T00:55:40.851ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae switch_location = Switch0
52492025-06-03T00:55:40.851ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
52502025-06-03T00:55:40.853ZINFOnexus (BackgroundTasks): VPC route manager sled afe89ed7-19b7-479b-81eb-05c988ebc66d push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
52512025-06-03T00:55:40.853ZDEBGnexus (BackgroundTasks): client request SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
52522025-06-03T00:55:40.853ZDEBGnexus (BackgroundTasks): client response SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "9dbc9b01-87e2-4b91-8d6a-f054020194d5", "date": "Tue, 03 Jun 2025 00:55:40 GMT"} })
52532025-06-03T00:55:40.855ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 349.447721ms iteration = 10 status = {}
52542025-06-03T00:55:40.987ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "4de5e3b9-7998-4001-bce8-868e459ab321", "date": "Tue, 03 Jun 2025 00:55:40 GMT"} })
52552025-06-03T00:55:40.987ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae switch = Switch0
52562025-06-03T00:55:40.987ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
52572025-06-03T00:55:40.987ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "a81b7af9-3641-4ab8-8909-374f6b6dc43a", "content-length": "682", "date": "Tue, 03 Jun 2025 00:55:40 GMT"} })
52582025-06-03T00:55:40.987ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
52592025-06-03T00:55:40.988ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
52602025-06-03T00:55:40.991ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
52612025-06-03T00:55:40.991ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
52622025-06-03T00:55:40.995ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
52632025-06-03T00:55:40.995ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
52642025-06-03T00:55:40.997ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
52652025-06-03T00:55:40.998ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
52662025-06-03T00:55:41.005ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 499.762929ms iteration = 10 status = {}
52672025-06-03T00:55:45.673ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 510 local_addr = [fd00:1122:3344:101::a]:63576 method = GET remote_addr = [fd00:1122:3344:101::d]:61315 req_id = 1354b441-610f-475b-b1af-afb75b8d9a24 response_code = 200 uri = /a2861392-5fd2-462b-84e5-6f36c716263a
52682025-06-03T00:55:50.505ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 29 reason = Timeout
52692025-06-03T00:55:50.505ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 29 reason = Timeout
52702025-06-03T00:55:50.505ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
52712025-06-03T00:55:50.505ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
52722025-06-03T00:55:50.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
52732025-06-03T00:55:50.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
52742025-06-03T00:55:50.507ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
52752025-06-03T00:55:50.508ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
52762025-06-03T00:55:50.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
52772025-06-03T00:55:50.509ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.907401ms iteration = 29 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
52782025-06-03T00:55:50.510ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
52792025-06-03T00:55:50.510ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
52802025-06-03T00:55:50.510ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
52812025-06-03T00:55:50.511ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
52822025-06-03T00:55:50.512ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
52832025-06-03T00:55:50.513ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
52842025-06-03T00:55:50.516ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 4ec93699-353f-4b9e-ae06-25355fdb069e, lookup_type: ById(4ec93699-353f-4b9e-ae06-25355fdb069e) } result = Ok(())
52852025-06-03T00:55:50.516ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
52862025-06-03T00:55:50.516ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
52872025-06-03T00:55:50.524ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 4ec93699-353f-4b9e-ae06-25355fdb069e original_time_created = 2025-06-03 00:51:07.059326 UTC target_id = 4ec93699-353f-4b9e-ae06-25355fdb069e time_created = 2025-06-03 00:51:07.059326 UTC
52882025-06-03T00:55:50.524ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 18.530435ms iteration = 29 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"4ec93699-353f-4b9e-ae06-25355fdb069e","time_created":"2025-06-03T00:51:07.059326Z"}
52892025-06-03T00:55:55.672ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 329 local_addr = [fd00:1122:3344:101::a]:63576 method = GET remote_addr = [fd00:1122:3344:101::d]:61315 req_id = fbd117e2-546d-425f-ac44-dbc2f2253178 response_code = 200 uri = /a2861392-5fd2-462b-84e5-6f36c716263a
52902025-06-03T00:56:00.504ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 30 reason = Timeout
52912025-06-03T00:56:00.505ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
52922025-06-03T00:56:00.505ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 30 reason = Timeout
52932025-06-03T00:56:00.505ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
52942025-06-03T00:56:00.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
52952025-06-03T00:56:00.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
52962025-06-03T00:56:00.506ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
52972025-06-03T00:56:00.507ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
52982025-06-03T00:56:00.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
52992025-06-03T00:56:00.509ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
53002025-06-03T00:56:00.509ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
53012025-06-03T00:56:00.509ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
53022025-06-03T00:56:00.511ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
53032025-06-03T00:56:00.511ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
53042025-06-03T00:56:00.513ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
53052025-06-03T00:56:00.515ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 4ec93699-353f-4b9e-ae06-25355fdb069e, lookup_type: ById(4ec93699-353f-4b9e-ae06-25355fdb069e) } result = Ok(())
53062025-06-03T00:56:00.515ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
53072025-06-03T00:56:00.515ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
53082025-06-03T00:56:00.526ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 4ec93699-353f-4b9e-ae06-25355fdb069e original_time_created = 2025-06-03 00:51:07.059326 UTC target_id = 4ec93699-353f-4b9e-ae06-25355fdb069e time_created = 2025-06-03 00:51:07.059326 UTC
53092025-06-03T00:56:00.526ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 21.502468ms iteration = 30 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"4ec93699-353f-4b9e-ae06-25355fdb069e","time_created":"2025-06-03T00:51:07.059326Z"}
53102025-06-03T00:56:00.544ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 39.328811ms iteration = 30 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
53112025-06-03T00:56:05.673ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 487 local_addr = [fd00:1122:3344:101::a]:63576 method = GET remote_addr = [fd00:1122:3344:101::d]:61315 req_id = 247bc66a-e079-4356-8abd-d433a960f409 response_code = 200 uri = /a2861392-5fd2-462b-84e5-6f36c716263a
53122025-06-03T00:56:10.504ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 31 reason = Timeout
53132025-06-03T00:56:10.504ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
53142025-06-03T00:56:10.504ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 6 reason = Timeout
53152025-06-03T00:56:10.504ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
53162025-06-03T00:56:10.504ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 6 reason = Timeout
53172025-06-03T00:56:10.504ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 6 reason = Timeout
53182025-06-03T00:56:10.504ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:53
53192025-06-03T00:56:10.504ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
53202025-06-03T00:56:10.504ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
53212025-06-03T00:56:10.504ZDEBGnexus (BackgroundTasks): activating background_task = webhook_deliverator iteration = 6 reason = Timeout
53222025-06-03T00:56:10.504ZDEBGnexus (BackgroundTasks): roles background_task = webhook_deliverator roles = RoleSet { roles: {} }
53232025-06-03T00:56:10.504ZDEBGnexus (BackgroundTasks): activating background_task = alert_dispatcher iteration = 6 reason = Timeout
53242025-06-03T00:56:10.504ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
53252025-06-03T00:56:10.504ZDEBGnexus (BackgroundTasks): activating background_task = service_firewall_rule_propagation iteration = 2 reason = Timeout
53262025-06-03T00:56:10.504ZDEBGnexus (service-firewall-rule-progator): starting background task for service firewall rule propagation background_task = service_firewall_rule_propagation
53272025-06-03T00:56:10.504ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
53282025-06-03T00:56:10.504ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_rendezvous iteration = 3 reason = Timeout
53292025-06-03T00:56:10.504ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
53302025-06-03T00:56:10.504ZDEBGnexus (BackgroundTasks): activating background_task = tuf_artifact_replication iteration = 2 reason = Timeout
53312025-06-03T00:56:10.504ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 6 reason = Timeout
53322025-06-03T00:56:10.504ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
53332025-06-03T00:56:10.504ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 7 reason = Timeout
53342025-06-03T00:56:10.504ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:85 reason = blueprint disabled target_id = 4ec93699-353f-4b9e-ae06-25355fdb069e
53352025-06-03T00:56:10.504ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 16.48\u{b5}s iteration = 7 status = {"enabled":false,"target_id":"4ec93699-353f-4b9e-ae06-25355fdb069e"}
53362025-06-03T00:56:10.504ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 6 reason = Timeout
53372025-06-03T00:56:10.504ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 8 reason = Timeout
53382025-06-03T00:56:10.505ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
53392025-06-03T00:56:10.505ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
53402025-06-03T00:56:10.505ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 6 reason = Timeout
53412025-06-03T00:56:10.505ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 8 reason = Timeout
53422025-06-03T00:56:10.505ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 6 reason = Timeout
53432025-06-03T00:56:10.505ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 6 reason = Timeout
53442025-06-03T00:56:10.505ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-06-03 00:46:10.505055475 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
53452025-06-03T00:56:10.505ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
53462025-06-03T00:56:10.505ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
53472025-06-03T00:56:10.505ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 11 reason = Timeout
53482025-06-03T00:56:10.505ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
53492025-06-03T00:56:10.505ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 11 reason = Timeout
53502025-06-03T00:56:10.505ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 11 reason = Timeout
53512025-06-03T00:56:10.505ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
53522025-06-03T00:56:10.505ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
53532025-06-03T00:56:10.505ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 11 reason = Timeout
53542025-06-03T00:56:10.505ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 11 reason = Timeout
53552025-06-03T00:56:10.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
53562025-06-03T00:56:10.505ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 11 reason = Timeout
53572025-06-03T00:56:10.505ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
53582025-06-03T00:56:10.505ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
53592025-06-03T00:56:10.505ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 11 reason = Timeout
53602025-06-03T00:56:10.505ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 11 reason = Timeout
53612025-06-03T00:56:10.505ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
53622025-06-03T00:56:10.505ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
53632025-06-03T00:56:10.505ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 11 reason = Timeout
53642025-06-03T00:56:10.505ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
53652025-06-03T00:56:10.505ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 12 reason = Timeout
53662025-06-03T00:56:10.505ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
53672025-06-03T00:56:10.505ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 11 reason = Timeout
53682025-06-03T00:56:10.505ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
53692025-06-03T00:56:10.505ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
53702025-06-03T00:56:10.505ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 11 reason = Timeout
53712025-06-03T00:56:10.505ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
53722025-06-03T00:56:10.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
53732025-06-03T00:56:10.505ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 6 reason = Timeout
53742025-06-03T00:56:10.505ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
53752025-06-03T00:56:10.505ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 31 reason = Timeout
53762025-06-03T00:56:10.505ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
53772025-06-03T00:56:10.505ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 11 reason = Timeout
53782025-06-03T00:56:10.505ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
53792025-06-03T00:56:10.505ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
53802025-06-03T00:56:10.505ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 11 reason = Timeout
53812025-06-03T00:56:10.505ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
53822025-06-03T00:56:10.505ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 11 reason = Timeout
53832025-06-03T00:56:10.505ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
53842025-06-03T00:56:10.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = alert_dispatcher resource = Database result = Ok(())
53852025-06-03T00:56:10.505ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 11 reason = Timeout
53862025-06-03T00:56:10.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
53872025-06-03T00:56:10.505ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 6 reason = Timeout
53882025-06-03T00:56:10.505ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
53892025-06-03T00:56:10.505ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
53902025-06-03T00:56:10.505ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 11 reason = Timeout
53912025-06-03T00:56:10.505ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
53922025-06-03T00:56:10.505ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
53932025-06-03T00:56:10.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
53942025-06-03T00:56:10.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
53952025-06-03T00:56:10.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
53962025-06-03T00:56:10.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
53972025-06-03T00:56:10.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
53982025-06-03T00:56:10.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = webhook_deliverator resource = Database result = Ok(())
53992025-06-03T00:56:10.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
54002025-06-03T00:56:10.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
54012025-06-03T00:56:10.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
54022025-06-03T00:56:10.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
54032025-06-03T00:56:10.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
54042025-06-03T00:56:10.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
54052025-06-03T00:56:10.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
54062025-06-03T00:56:10.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
54072025-06-03T00:56:10.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
54082025-06-03T00:56:10.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
54092025-06-03T00:56:10.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
54102025-06-03T00:56:10.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
54112025-06-03T00:56:10.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
54122025-06-03T00:56:10.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
54132025-06-03T00:56:10.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
54142025-06-03T00:56:10.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
54152025-06-03T00:56:10.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
54162025-06-03T00:56:10.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
54172025-06-03T00:56:10.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
54182025-06-03T00:56:10.506ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("9feb16c9-ff60-43e8-917b-965124f46e0d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("aa124775-31de-4648-981a-4af500268b66.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("aa124775-31de-4648-981a-4af500268b66.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::9))) }], valid_until: Instant { tv_sec: 2697, tv_nsec: 730984208 } })
54192025-06-03T00:56:10.506ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("241573bc-467c-4802-9f4b-f7c82da4e12e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("64fc0a69-d4b8-4f6c-9a07-77a4cfa94962.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("ceb74144-4b6b-4b3f-bc80-eac2d75f3bc9.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("ceb74144-4b6b-4b3f-bc80-eac2d75f3bc9.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:2::1))) }], valid_until: Instant { tv_sec: 2697, tv_nsec: 731013959 } })
54202025-06-03T00:56:10.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
54212025-06-03T00:56:10.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
54222025-06-03T00:56:10.506ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
54232025-06-03T00:56:10.506ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.718983ms iteration = 6 status = {"addresses":["[fd00:1122:3344:101::8]:5353","[fd00:1122:3344:101::9]:5353"]}
54242025-06-03T00:56:10.506ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } dns_group = internal
54252025-06-03T00:56:10.506ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.725054ms iteration = 6 status = {"addresses":["[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:3::1]:5353","[fd00:1122:3344:2::1]:5353"]}
54262025-06-03T00:56:10.511ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
54272025-06-03T00:56:10.511ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
54282025-06-03T00:56:10.514ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 9.113873ms iteration = 11 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
54292025-06-03T00:56:10.514ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54302025-06-03T00:56:10.514ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54312025-06-03T00:56:10.515ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54322025-06-03T00:56:10.517ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 13.031425ms iteration = 6 status = {"errors":[],"found_port_ok":[]}
54332025-06-03T00:56:10.518ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
54342025-06-03T00:56:10.518ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
54352025-06-03T00:56:10.518ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
54362025-06-03T00:56:10.519ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
54372025-06-03T00:56:10.519ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Fleet result = Ok(())
54382025-06-03T00:56:10.519ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
54392025-06-03T00:56:10.520ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
54402025-06-03T00:56:10.520ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
54412025-06-03T00:56:10.520ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Inventory result = Ok(())
54422025-06-03T00:56:10.520ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
54432025-06-03T00:56:10.520ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
54442025-06-03T00:56:10.525ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-06-03 00:46:10.505055475 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
54452025-06-03T00:56:10.525ZINFOnexus (BackgroundTasks): all webhook delivery tasks completed background_task = webhook_deliverator file = nexus/src/app/background/tasks/webhook_deliverator.rs:202 num_receivers = 0
54462025-06-03T00:56:10.525ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 20.62003ms iteration = 6 status = {"expiration":"2025-06-03T00:46:10.505055475Z","pruned":[]}
54472025-06-03T00:56:10.525ZDEBGnexus (BackgroundTasks): activation complete background_task = webhook_deliverator elapsed = 20.892418ms iteration = 6 status = {"by_rx":{},"error":null}
54482025-06-03T00:56:10.531ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
54492025-06-03T00:56:10.531ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
54502025-06-03T00:56:10.533ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
54512025-06-03T00:56:10.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
54522025-06-03T00:56:10.534ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54532025-06-03T00:56:10.536ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54542025-06-03T00:56:10.539ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54552025-06-03T00:56:10.539ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
54562025-06-03T00:56:10.539ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
54572025-06-03T00:56:10.539ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
54582025-06-03T00:56:10.539ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54592025-06-03T00:56:10.540ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
54602025-06-03T00:56:10.540ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
54612025-06-03T00:56:10.540ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
54622025-06-03T00:56:10.540ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
54632025-06-03T00:56:10.540ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
54642025-06-03T00:56:10.540ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
54652025-06-03T00:56:10.542ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54662025-06-03T00:56:10.544ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
54672025-06-03T00:56:10.544ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
54682025-06-03T00:56:10.544ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
54692025-06-03T00:56:10.545ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
54702025-06-03T00:56:10.545ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
54712025-06-03T00:56:10.546ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
54722025-06-03T00:56:10.555ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54732025-06-03T00:56:10.555ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
54742025-06-03T00:56:10.556ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
54752025-06-03T00:56:10.558ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
54762025-06-03T00:56:10.558ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
54772025-06-03T00:56:10.558ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
54782025-06-03T00:56:10.560ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-afe89ed7-19b7-479b-81eb-05c988ebc66d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-afe89ed7-19b7-479b-81eb-05c988ebc66d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 2697, tv_nsec: 785121029 } })
54792025-06-03T00:56:10.560ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1179
54802025-06-03T00:56:10.562ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-afe89ed7-19b7-479b-81eb-05c988ebc66d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-afe89ed7-19b7-479b-81eb-05c988ebc66d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 2697, tv_nsec: 787664958 } })
54812025-06-03T00:56:10.563ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1179
54822025-06-03T00:56:10.574ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
54832025-06-03T00:56:10.574ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
54842025-06-03T00:56:10.583ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
54852025-06-03T00:56:10.583ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
54862025-06-03T00:56:10.586ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
54872025-06-03T00:56:10.587ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
54882025-06-03T00:56:10.591ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
54892025-06-03T00:56:10.591ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = alert_dispatcher resource = Database result = Ok(())
54902025-06-03T00:56:10.597ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
54912025-06-03T00:56:10.598ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
54922025-06-03T00:56:10.599ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
54932025-06-03T00:56:10.599ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
54942025-06-03T00:56:10.603ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
54952025-06-03T00:56:10.603ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
54962025-06-03T00:56:10.603ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
54972025-06-03T00:56:10.603ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
54982025-06-03T00:56:10.607ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
54992025-06-03T00:56:10.607ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
55002025-06-03T00:56:10.609ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
55012025-06-03T00:56:10.611ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
55022025-06-03T00:56:10.611ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
55032025-06-03T00:56:10.612ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
55042025-06-03T00:56:10.612ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
55052025-06-03T00:56:10.612ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
55062025-06-03T00:56:10.616ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
55072025-06-03T00:56:10.618ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-afe89ed7-19b7-479b-81eb-05c988ebc66d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-afe89ed7-19b7-479b-81eb-05c988ebc66d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 2697, tv_nsec: 843792181 } })
55082025-06-03T00:56:10.619ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1179 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae
55092025-06-03T00:56:10.620ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
55102025-06-03T00:56:10.626ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
55112025-06-03T00:56:10.629ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
55122025-06-03T00:56:10.629ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
55132025-06-03T00:56:10.630ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
55142025-06-03T00:56:10.633ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 127.899932ms iteration = 11 status = {"errors":[],"finish_invoked_ok":[]}
55152025-06-03T00:56:10.633ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
55162025-06-03T00:56:10.633ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:167 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
55172025-06-03T00:56:10.633ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 127.784259ms iteration = 6 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
55182025-06-03T00:56:10.633ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
55192025-06-03T00:56:10.634ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
55202025-06-03T00:56:10.636ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 131.261506ms iteration = 11 status = {"errors":[],"requests_created_ok":[]}
55212025-06-03T00:56:10.637ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 132.866456ms iteration = 31 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
55222025-06-03T00:56:10.637ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 132.638129ms iteration = 11 status = {"errors":[],"garbage_collect_requested":[]}
55232025-06-03T00:56:10.637ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Fleet result = Ok(())
55242025-06-03T00:56:10.637ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
55252025-06-03T00:56:10.638ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
55262025-06-03T00:56:10.639ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
55272025-06-03T00:56:10.639ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
55282025-06-03T00:56:10.639ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
55292025-06-03T00:56:10.639ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
55302025-06-03T00:56:10.640ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
55312025-06-03T00:56:10.640ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
55322025-06-03T00:56:10.640ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:137 found = 0 not_ready_to_be_deleted = 0
55332025-06-03T00:56:10.640ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 135.793726ms iteration = 6 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
55342025-06-03T00:56:10.641ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
55352025-06-03T00:56:10.641ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
55362025-06-03T00:56:10.641ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
55372025-06-03T00:56:10.642ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
55382025-06-03T00:56:10.642ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
55392025-06-03T00:56:10.642ZDEBGnexus (BackgroundTasks): activation complete background_task = alert_dispatcher elapsed = 137.437818ms iteration = 6 status = {"dispatched":[],"errors":[],"glob_version":"144.0.0","globs_reprocessed":{},"no_receivers":[]}
55402025-06-03T00:56:10.644ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
55412025-06-03T00:56:10.644ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
55422025-06-03T00:56:10.645ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
55432025-06-03T00:56:10.647ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 4ec93699-353f-4b9e-ae06-25355fdb069e, lookup_type: ById(4ec93699-353f-4b9e-ae06-25355fdb069e) } result = Ok(())
55442025-06-03T00:56:10.647ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
55452025-06-03T00:56:10.647ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
55462025-06-03T00:56:10.647ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
55472025-06-03T00:56:10.647ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
55482025-06-03T00:56:10.647ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
55492025-06-03T00:56:10.650ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
55502025-06-03T00:56:10.650ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
55512025-06-03T00:56:10.650ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
55522025-06-03T00:56:10.650ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
55532025-06-03T00:56:10.650ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
55542025-06-03T00:56:10.650ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
55552025-06-03T00:56:10.651ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
55562025-06-03T00:56:10.651ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
55572025-06-03T00:56:10.654ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
55582025-06-03T00:56:10.655ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
55592025-06-03T00:56:10.656ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:547 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
55602025-06-03T00:56:10.656ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 150.996249ms iteration = 11 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
55612025-06-03T00:56:10.669ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 4ec93699-353f-4b9e-ae06-25355fdb069e original_time_created = 2025-06-03 00:51:07.059326 UTC target_id = 4ec93699-353f-4b9e-ae06-25355fdb069e time_created = 2025-06-03 00:51:07.059326 UTC
55622025-06-03T00:56:10.669ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 164.533459ms iteration = 31 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"4ec93699-353f-4b9e-ae06-25355fdb069e","time_created":"2025-06-03T00:51:07.059326Z"}
55632025-06-03T00:56:10.687ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
55642025-06-03T00:56:10.689ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
55652025-06-03T00:56:10.689ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
55662025-06-03T00:56:10.689ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
55672025-06-03T00:56:10.694ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
55682025-06-03T00:56:10.695ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
55692025-06-03T00:56:10.695ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
55702025-06-03T00:56:10.695ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
55712025-06-03T00:56:10.695ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
55722025-06-03T00:56:10.702ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
55732025-06-03T00:56:10.702ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
55742025-06-03T00:56:10.703ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 198.233725ms iteration = 11 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
55752025-06-03T00:56:10.703ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
55762025-06-03T00:56:10.703ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
55772025-06-03T00:56:10.703ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
55782025-06-03T00:56:10.703ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
55792025-06-03T00:56:10.703ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
55802025-06-03T00:56:10.703ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
55812025-06-03T00:56:10.703ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
55822025-06-03T00:56:10.708ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
55832025-06-03T00:56:10.708ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
55842025-06-03T00:56:10.712ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
55852025-06-03T00:56:10.712ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
55862025-06-03T00:56:10.712ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 207.175553ms iteration = 11 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
55872025-06-03T00:56:10.716ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
55882025-06-03T00:56:10.717ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
55892025-06-03T00:56:10.719ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1187 zone_address = fd00:1122:3344:101::2
55902025-06-03T00:56:10.719ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
55912025-06-03T00:56:10.720ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "0da17f1d-6180-4f44-a17f-e66d2221c4ae", "content-length": "26", "date": "Tue, 03 Jun 2025 00:56:10 GMT"} })
55922025-06-03T00:56:10.720ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1190 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
55932025-06-03T00:56:10.722ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1224 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
55942025-06-03T00:56:10.773ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
55952025-06-03T00:56:10.773ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "10cd7c07-94e4-499d-8b17-f453a12b1807", "date": "Tue, 03 Jun 2025 00:56:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
55962025-06-03T00:56:10.773ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
55972025-06-03T00:56:10.775ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "1f6c4bc4-e212-4795-bcd0-5bcbe7f1a665", "date": "Tue, 03 Jun 2025 00:56:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
55982025-06-03T00:56:10.775ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
55992025-06-03T00:56:10.777ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
56002025-06-03T00:56:10.777ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 272.481871ms iteration = 8 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
56012025-06-03T00:56:10.795ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 289.933873ms iteration = 12 status = {"physical_disks_added":0}
56022025-06-03T00:56:10.812ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
56032025-06-03T00:56:10.812ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
56042025-06-03T00:56:10.814ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
56052025-06-03T00:56:10.814ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
56062025-06-03T00:56:10.815ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
56072025-06-03T00:56:10.817ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
56082025-06-03T00:56:10.818ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
56092025-06-03T00:56:10.820ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
56102025-06-03T00:56:10.820ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
56112025-06-03T00:56:10.821ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
56122025-06-03T00:56:10.822ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
56132025-06-03T00:56:10.822ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
56142025-06-03T00:56:10.822ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
56152025-06-03T00:56:10.825ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
56162025-06-03T00:56:10.827ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
56172025-06-03T00:56:10.827ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
56182025-06-03T00:56:10.827ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
56192025-06-03T00:56:10.840ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1187 zone_address = fd00:1122:3344:101::2
56202025-06-03T00:56:10.840ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
56212025-06-03T00:56:10.840ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "6417501f-1adf-428a-a288-626d9912ede9", "content-length": "26", "date": "Tue, 03 Jun 2025 00:56:10 GMT"} })
56222025-06-03T00:56:10.840ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1190 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
56232025-06-03T00:56:10.843ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1224 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
56242025-06-03T00:56:10.852ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
56252025-06-03T00:56:10.852ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
56262025-06-03T00:56:10.854ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
56272025-06-03T00:56:10.857ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
56282025-06-03T00:56:10.857ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
56292025-06-03T00:56:10.857ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
56302025-06-03T00:56:10.864ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
56312025-06-03T00:56:10.866ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
56322025-06-03T00:56:10.866ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
56332025-06-03T00:56:10.867ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
56342025-06-03T00:56:10.870ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
56352025-06-03T00:56:10.872ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
56362025-06-03T00:56:10.872ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 3ef110b4-9400-4c37-8d7f-5552c8a94db0 dns_zone_name = oxide.test found_so_far = 0 version = 2
56372025-06-03T00:56:10.872ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
56382025-06-03T00:56:10.872ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
56392025-06-03T00:56:10.873ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
56402025-06-03T00:56:10.874ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
56412025-06-03T00:56:10.874ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
56422025-06-03T00:56:10.874ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
56432025-06-03T00:56:10.876ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
56442025-06-03T00:56:10.876ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
56452025-06-03T00:56:10.876ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
56462025-06-03T00:56:10.877ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
56472025-06-03T00:56:10.878ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
56482025-06-03T00:56:10.878ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 373.578261ms iteration = 11 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
56492025-06-03T00:56:10.878ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Fleet result = Ok(())
56502025-06-03T00:56:10.878ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
56512025-06-03T00:56:10.879ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
56522025-06-03T00:56:10.880ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
56532025-06-03T00:56:10.880ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
56542025-06-03T00:56:10.880ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
56552025-06-03T00:56:10.881ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
56562025-06-03T00:56:10.881ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
56572025-06-03T00:56:10.881ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
56582025-06-03T00:56:10.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
56592025-06-03T00:56:10.883ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1187 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae zone_address = fd00:1122:3344:101::2
56602025-06-03T00:56:10.883ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
56612025-06-03T00:56:10.883ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "3696113a-c8e7-4964-a1f0-546cf1ae1e9d", "content-length": "26", "date": "Tue, 03 Jun 2025 00:56:10 GMT"} })
56622025-06-03T00:56:10.883ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1190 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
56632025-06-03T00:56:10.885ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1224 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae
56642025-06-03T00:56:10.900ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
56652025-06-03T00:56:10.900ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
56662025-06-03T00:56:10.905ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 400.394233ms iteration = 11 status = {}
56672025-06-03T00:56:10.959ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
56682025-06-03T00:56:10.960ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "798433ad-dbb0-4c06-9789-ed4f8c2ba1cd", "date": "Tue, 03 Jun 2025 00:56:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
56692025-06-03T00:56:10.960ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
56702025-06-03T00:56:10.962ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "8dfc3e14-d9a9-499a-849d-9467fce71689", "date": "Tue, 03 Jun 2025 00:56:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
56712025-06-03T00:56:10.963ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
56722025-06-03T00:56:10.965ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "f0424fac-31d3-47a5-9a0e-739120e7efc1", "date": "Tue, 03 Jun 2025 00:56:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
56732025-06-03T00:56:10.965ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
56742025-06-03T00:56:10.967ZDEBGnexus (BackgroundTasks): client request SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
56752025-06-03T00:56:10.967ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
56762025-06-03T00:56:10.968ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 463.063079ms iteration = 8 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
56772025-06-03T00:56:10.980ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 475.171245ms iteration = 11 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
56782025-06-03T00:56:10.987ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
56792025-06-03T00:56:10.987ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
56802025-06-03T00:56:10.993ZINFOnexus (BackgroundTasks): instance reincarnation completed successfully background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:113 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0
56812025-06-03T00:56:10.993ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 489.072626ms iteration = 6 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
56822025-06-03T00:56:11.008ZINFOnexus (BackgroundTasks): VPC route manager sled afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
56832025-06-03T00:56:11.008ZDEBGnexus (BackgroundTasks): client request SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
56842025-06-03T00:56:11.011ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
56852025-06-03T00:56:11.012ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
56862025-06-03T00:56:11.013ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
56872025-06-03T00:56:11.013ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
56882025-06-03T00:56:11.014ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
56892025-06-03T00:56:11.015ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
56902025-06-03T00:56:11.016ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
56912025-06-03T00:56:11.019ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
56922025-06-03T00:56:11.019ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
56932025-06-03T00:56:11.019ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
56942025-06-03T00:56:11.024ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
56952025-06-03T00:56:11.025ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
56962025-06-03T00:56:11.028ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
56972025-06-03T00:56:11.028ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 523.138624ms iteration = 11 status = null
56982025-06-03T00:56:11.029ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
56992025-06-03T00:56:11.030ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
57002025-06-03T00:56:11.030ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
57012025-06-03T00:56:11.031ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 3ef110b4-9400-4c37-8d7f-5552c8a94db0 dns_zone_name = oxide.test found_so_far = 4 version = 2
57022025-06-03T00:56:11.031ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
57032025-06-03T00:56:11.031ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2025-06-03 00:51:38.567192 UTC dns_group = external generation = 2
57042025-06-03T00:56:11.031ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 526.147128ms iteration = 6 status = {"generation":2}
57052025-06-03T00:56:11.031ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
57062025-06-03T00:56:11.031ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
57072025-06-03T00:56:11.031ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
57082025-06-03T00:56:11.032ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Fleet result = Ok(())
57092025-06-03T00:56:11.032ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
57102025-06-03T00:56:11.032ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
57112025-06-03T00:56:11.033ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
57122025-06-03T00:56:11.033ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
57132025-06-03T00:56:11.035ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
57142025-06-03T00:56:11.037ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
57152025-06-03T00:56:11.037ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
57162025-06-03T00:56:11.037ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
57172025-06-03T00:56:11.038ZDEBGnexus (BackgroundTasks): client request SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = tuf_artifact_replication body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/artifacts-config
57182025-06-03T00:56:11.038ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: afe89ed7-19b7-479b-81eb-05c988ebc66d, lookup_type: ById(afe89ed7-19b7-479b-81eb-05c988ebc66d) } result = Ok(())
57192025-06-03T00:56:11.038ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
57202025-06-03T00:56:11.038ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
57212025-06-03T00:56:11.040ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
57222025-06-03T00:56:11.040ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
57232025-06-03T00:56:11.040ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
57242025-06-03T00:56:11.041ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 535.998393ms iteration = 11 status = 0
57252025-06-03T00:56:11.045ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
57262025-06-03T00:56:11.045ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
57272025-06-03T00:56:11.047ZINFOnexus (BackgroundTasks): ensured all Crucible datasets present in inventory have database records background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/crucible_dataset.rs:113 num_already_exist = 9 num_inserted = 0 num_not_in_inventory = 0
57282025-06-03T00:56:11.049ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_rendezvous elapsed = 544.733715ms iteration = 3 status = {"blueprint_id":"4ec93699-353f-4b9e-ae06-25355fdb069e","inventory_collection_id":"0105612f-4266-4b85-968a-ef7f273e6da5","stats":{"crucible_dataset":{"num_already_exist":9,"num_inserted":0,"num_not_in_inventory":0},"debug_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0}}}
57292025-06-03T00:56:11.049ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
57302025-06-03T00:56:11.050ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
57312025-06-03T00:56:11.051ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
57322025-06-03T00:56:11.051ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
57332025-06-03T00:56:11.053ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
57342025-06-03T00:56:11.053ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
57352025-06-03T00:56:11.055ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
57362025-06-03T00:56:11.055ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
57372025-06-03T00:56:11.057ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
57382025-06-03T00:56:11.057ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
57392025-06-03T00:56:11.059ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
57402025-06-03T00:56:11.062ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
57412025-06-03T00:56:11.062ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 9e79f9b1-a4de-4f9d-a901-16cc4f79e15d dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
57422025-06-03T00:56:11.062ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
57432025-06-03T00:56:11.062ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
57442025-06-03T00:56:11.062ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
57452025-06-03T00:56:11.062ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:372
57462025-06-03T00:56:11.062ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
57472025-06-03T00:56:11.062ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 557.823152ms iteration = 11 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
57482025-06-03T00:56:11.063ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
57492025-06-03T00:56:11.064ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
57502025-06-03T00:56:11.065ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
57512025-06-03T00:56:11.065ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
57522025-06-03T00:56:11.066ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
57532025-06-03T00:56:11.066ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
57542025-06-03T00:56:11.066ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
57552025-06-03T00:56:11.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
57562025-06-03T00:56:11.067ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
57572025-06-03T00:56:11.069ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 9e79f9b1-a4de-4f9d-a901-16cc4f79e15d dns_zone_name = control-plane.oxide.internal found_so_far = 59 version = 1
57582025-06-03T00:56:11.069ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
57592025-06-03T00:56:11.069ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2025-06-03 00:51:38.152773 UTC dns_group = internal generation = 1
57602025-06-03T00:56:11.069ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 564.062535ms iteration = 6 status = {"generation":1}
57612025-06-03T00:56:11.069ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
57622025-06-03T00:56:11.069ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
57632025-06-03T00:56:11.070ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-06-03T00:51:07.585932Z, time_modified: 2025-06-03T00:51:07.585932Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: ea0ef520-6bf3-4fbd-9ff0-f3698b48b749, db_silo: Silo { identity: SiloIdentity { id: ea0ef520-6bf3-4fbd-9ff0-f3698b48b749, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-06-03T00:51:38.523369Z, time_modified: 2025-06-03T00:51:38.523369Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "bcda8a9f16e63c3cec49f9759478b9f335a661639158b0a9026f69b846d4e127" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: ea0ef520-6bf3-4fbd-9ff0-f3698b48b749, db_silo: Silo { identity: SiloIdentity { id: ea0ef520-6bf3-4fbd-9ff0-f3698b48b749, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-06-03T00:51:38.523369Z, time_modified: 2025-06-03T00:51:38.523369Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "bcda8a9f16e63c3cec49f9759478b9f335a661639158b0a9026f69b846d4e127" }] }) }
57642025-06-03T00:56:11.070ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 565.835061ms iteration = 6 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"ea0ef520-6bf3-4fbd-9ff0-f3698b48b749","tls_certs":["bcda8a9f16e63c3cec49f9759478b9f335a661639158b0a9026f69b846d4e127"]}},"default_endpoint":{"silo_id":"ea0ef520-6bf3-4fbd-9ff0-f3698b48b749","tls_certs":["bcda8a9f16e63c3cec49f9759478b9f335a661639158b0a9026f69b846d4e127"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
57652025-06-03T00:56:11.071ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
57662025-06-03T00:56:11.071ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
57672025-06-03T00:56:11.073ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
57682025-06-03T00:56:11.073ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
57692025-06-03T00:56:11.074ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
57702025-06-03T00:56:11.075ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
57712025-06-03T00:56:11.076ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
57722025-06-03T00:56:11.080ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
57732025-06-03T00:56:11.080ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
57742025-06-03T00:56:11.080ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
57752025-06-03T00:56:11.081ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
57762025-06-03T00:56:11.081ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
57772025-06-03T00:56:11.084ZDEBGnexus (BackgroundTasks): client response SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "e0f620a3-7b73-4caf-9675-29aaf1ffddc8", "content-length": "691", "date": "Tue, 03 Jun 2025 00:56:10 GMT"} })
57782025-06-03T00:56:11.084ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
57792025-06-03T00:56:11.084ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
57802025-06-03T00:56:11.084ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
57812025-06-03T00:56:11.084ZDEBGnexus (BackgroundTasks): client response SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "03b33c92-35e1-4e94-9221-83ea5a7aad2d", "content-length": "372", "date": "Tue, 03 Jun 2025 00:56:11 GMT"} })
57822025-06-03T00:56:11.084ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
57832025-06-03T00:56:11.084ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
57842025-06-03T00:56:11.086ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 581.63078ms iteration = 11 status = {}
57852025-06-03T00:56:11.088ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
57862025-06-03T00:56:11.088ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
57872025-06-03T00:56:11.090ZDEBGnexus (BackgroundTasks): client response SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = tuf_artifact_replication result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/artifacts-config", status: 204, headers: {"x-request-id": "50a0762f-5505-474b-a6b0-d56df9aa06d0", "date": "Tue, 03 Jun 2025 00:56:11 GMT"} })
57882025-06-03T00:56:11.091ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: ec5e33f9-c697-46e2-a93e-a745336bf9dc, rack_id: d6831ba0-55f0-4191-8dc2-ea15e55ab0ae, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(9378a284-0ee9-48e2-80a0-3bcaadb1c4e2) } rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae switch_location = Switch0
57892025-06-03T00:56:11.091ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
57902025-06-03T00:56:11.091ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
57912025-06-03T00:56:11.093ZDEBGnexus (BackgroundTasks): client request SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = tuf_artifact_replication body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/artifacts
57922025-06-03T00:56:11.093ZDEBGnexus (BackgroundTasks): client response SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = tuf_artifact_replication result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/artifacts", status: 200, headers: {"content-type": "application/json", "x-request-id": "3d9a1fdb-5bb5-4cd7-b85c-ab2c374150fe", "content-length": "26", "date": "Tue, 03 Jun 2025 00:56:11 GMT"} })
57932025-06-03T00:56:11.096ZINFOnexus (BackgroundTasks): Successfully got artifact list background_task = tuf_artifact_replication file = nexus/src/app/background/tasks/tuf_artifact_replication.rs:673 sled = http://[fd00:1122:3344:101::1]:12345
57942025-06-03T00:56:11.098ZDEBGnexus (BackgroundTasks): activation complete background_task = tuf_artifact_replication elapsed = 593.352134ms iteration = 2 status = {"generation":1,"last_run_counters":{"copy_err":0,"copy_ok":0,"list_err":0,"list_ok":1,"put_config_err":0,"put_config_ok":1,"put_err":0,"put_ok":0},"lifetime_counters":{"copy_err":0,"copy_ok":0,"list_err":0,"list_ok":1,"put_config_err":0,"put_config_ok":1,"put_err":0,"put_ok":0},"local_repos":0,"request_debug_ringbuf":[{"generation":1,"operation":"put_config","target_sled":"afe89ed7-19b7-479b-81eb-05c988ebc66d","time":"2025-06-03T00:56:11.090627419Z"},{"operation":"list","target_sled":"afe89ed7-19b7-479b-81eb-05c988ebc66d","time":"2025-06-03T00:56:11.096698767Z"}]}
57952025-06-03T00:56:11.105ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
57962025-06-03T00:56:11.105ZDEBGnexus (BackgroundTasks): client request SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
57972025-06-03T00:56:11.109ZDEBGnexus (BackgroundTasks): client response SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "914421b1-9323-4f6c-bd1f-6291cec1d39c", "date": "Tue, 03 Jun 2025 00:56:10 GMT"} })
57982025-06-03T00:56:11.109ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
57992025-06-03T00:56:11.110ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
58002025-06-03T00:56:11.112ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
58012025-06-03T00:56:11.112ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
58022025-06-03T00:56:11.116ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
58032025-06-03T00:56:11.116ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1832 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
58042025-06-03T00:56:11.116ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1859 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 9378a284-0ee9-48e2-80a0-3bcaadb1c4e2
58052025-06-03T00:56:11.116ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
58062025-06-03T00:56:11.116ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
58072025-06-03T00:56:11.116ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
58082025-06-03T00:56:11.116ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
58092025-06-03T00:56:11.116ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
58102025-06-03T00:56:11.116ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
58112025-06-03T00:56:11.116ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2006 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
58122025-06-03T00:56:11.117ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:408 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
58132025-06-03T00:56:11.117ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2042 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
58142025-06-03T00:56:11.117ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae
58152025-06-03T00:56:11.117ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
58162025-06-03T00:56:11.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
58172025-06-03T00:56:11.120ZINFOnexus (BackgroundTasks): VPC route manager sled afe89ed7-19b7-479b-81eb-05c988ebc66d push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
58182025-06-03T00:56:11.120ZDEBGnexus (BackgroundTasks): client request SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
58192025-06-03T00:56:11.120ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae switch_location = Switch0
58202025-06-03T00:56:11.120ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
58212025-06-03T00:56:11.124ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
58222025-06-03T00:56:11.124ZDEBGnexus (BackgroundTasks): client response SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "8f6c82d7-107b-4ff4-becd-5f399d0d1902", "date": "Tue, 03 Jun 2025 00:56:11 GMT"} })
58232025-06-03T00:56:11.125ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
58242025-06-03T00:56:11.126ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
58252025-06-03T00:56:11.127ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 621.84314ms iteration = 11 status = {}
58262025-06-03T00:56:11.130ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
58272025-06-03T00:56:11.130ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
58282025-06-03T00:56:11.130ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
58292025-06-03T00:56:11.137ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
58302025-06-03T00:56:11.137ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
58312025-06-03T00:56:11.138ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
58322025-06-03T00:56:11.139ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
58332025-06-03T00:56:11.140ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
58342025-06-03T00:56:11.141ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
58352025-06-03T00:56:11.143ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
58362025-06-03T00:56:11.143ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
58372025-06-03T00:56:11.145ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
58382025-06-03T00:56:11.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
58392025-06-03T00:56:11.146ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
58402025-06-03T00:56:11.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
58412025-06-03T00:56:11.147ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
58422025-06-03T00:56:11.147ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
58432025-06-03T00:56:11.149ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
58442025-06-03T00:56:11.153ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
58452025-06-03T00:56:11.153ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
58462025-06-03T00:56:11.153ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
58472025-06-03T00:56:11.154ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
58482025-06-03T00:56:11.154ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
58492025-06-03T00:56:11.156ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
58502025-06-03T00:56:11.156ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
58512025-06-03T00:56:11.158ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
58522025-06-03T00:56:11.163ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
58532025-06-03T00:56:11.163ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
58542025-06-03T00:56:11.163ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
58552025-06-03T00:56:11.170ZDEBGnexus (service-firewall-rule-progator): resolved names for firewall rules background_task = service_firewall_rule_propagation instance_interfaces = {} subnet_interfaces = {Name("external-dns"): [NetworkInterface { id: 3aad1b77-42d7-48c8-9121-0ef5c4effed9, kind: Service { id: aa124775-31de-4648-981a-4af500268b66 }, name: Name("external-dns-aa124775-31de-4648-981a-4af500268b66"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 180, 88])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: a9971f4d-e41a-4de5-9ec6-26e6471eb4a4, kind: Service { id: 9feb16c9-ff60-43e8-917b-965124f46e0d }, name: Name("external-dns-9feb16c9-ff60-43e8-917b-965124f46e0d"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 190, 11])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], Name("nexus"): [NetworkInterface { id: 68a89d5e-a312-4992-a0a9-921a41d0d3dd, kind: Service { id: 185a1d3b-437a-4bdb-baa0-fc5d8728a3a8 }, name: Name("nexus-185a1d3b-437a-4bdb-baa0-fc5d8728a3a8"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 199, 152])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 68ff820c-70a6-4b17-95d5-d3c31f7b4c51, kind: Service { id: fe12ffc1-f2ac-4bf3-9672-fc185f82c59b }, name: Name("nexus-fe12ffc1-f2ac-4bf3-9672-fc185f82c59b"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 216, 245])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: ece06ff8-68bc-4632-b601-b8b26de7e08d, kind: Service { id: a2861392-5fd2-462b-84e5-6f36c716263a }, name: Name("nexus-a2861392-5fd2-462b-84e5-6f36c716263a"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 243, 2])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }]} subnet_networks = {Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })], Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })]} vpc_interfaces = {}
58562025-06-03T00:56:11.170ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
58572025-06-03T00:56:11.171ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
58582025-06-03T00:56:11.172ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
58592025-06-03T00:56:11.174ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Fleet result = Ok(())
58602025-06-03T00:56:11.174ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
58612025-06-03T00:56:11.174ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
58622025-06-03T00:56:11.177ZDEBGnexus (service-firewall-rule-progator): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified. background_task = service_firewall_rule_propagation
58632025-06-03T00:56:11.180ZDEBGnexus (service-firewall-rule-progator): resolved firewall rules for sled agents background_task = service_firewall_rule_propagation sled_agent_rules = [ResolvedVpcFirewallRule { action: Allow, direction: Inbound, filter_hosts: None, filter_ports: Some([L4PortRange("53")]), filter_protocols: Some([Udp]), priority: 65534, status: Enabled, targets: [NetworkInterface { id: 3aad1b77-42d7-48c8-9121-0ef5c4effed9, kind: Service { id: aa124775-31de-4648-981a-4af500268b66 }, name: Name("external-dns-aa124775-31de-4648-981a-4af500268b66"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 180, 88])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: a9971f4d-e41a-4de5-9ec6-26e6471eb4a4, kind: Service { id: 9feb16c9-ff60-43e8-917b-965124f46e0d }, name: Name("external-dns-9feb16c9-ff60-43e8-917b-965124f46e0d"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 190, 11])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }] }, ResolvedVpcFirewallRule { action: Allow, direction: Inbound, filter_hosts: None, filter_ports: Some([L4PortRange("80"), L4PortRange("443")]), filter_protocols: Some([Tcp]), priority: 65534, status: Enabled, targets: [NetworkInterface { id: 68a89d5e-a312-4992-a0a9-921a41d0d3dd, kind: Service { id: 185a1d3b-437a-4bdb-baa0-fc5d8728a3a8 }, name: Name("nexus-185a1d3b-437a-4bdb-baa0-fc5d8728a3a8"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 199, 152])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 68ff820c-70a6-4b17-95d5-d3c31f7b4c51, kind: Service { id: fe12ffc1-f2ac-4bf3-9672-fc185f82c59b }, name: Name("nexus-fe12ffc1-f2ac-4bf3-9672-fc185f82c59b"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 216, 245])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: ece06ff8-68bc-4632-b601-b8b26de7e08d, kind: Service { id: a2861392-5fd2-462b-84e5-6f36c716263a }, name: Name("nexus-a2861392-5fd2-462b-84e5-6f36c716263a"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 243, 2])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }] }]
58642025-06-03T00:56:11.180ZDEBGnexus (service-firewall-rule-progator): resolved 2 rules for sleds background_task = service_firewall_rule_propagation
58652025-06-03T00:56:11.262ZDEBGnexus (service-firewall-rule-progator): resolved sleds for vpc oxide-services background_task = service_firewall_rule_propagation vpc_to_sled = [Sled { identity: SledIdentity { id: afe89ed7-19b7-479b-81eb-05c988ebc66d, time_created: 2025-06-03T00:51:37.177791Z, time_modified: 2025-06-03T00:51:37.177791Z }, time_deleted: None, rcgen: Generation(Generation(19)), rack_id: d6831ba0-55f0-4191-8dc2-ea15e55ab0ae, is_scrimlet: true, serial_number: "sock", part_number: "i86pc", revision: SqlU32(0), usable_hardware_threads: SqlU32(48), usable_physical_ram: ByteCount(ByteCount(102952026112)), reservoir_size: ByteCount(ByteCount(56094621696)), ip: fd00:1122:3344:101::1, port: SqlU16(12345), last_used_address: fd00:1122:3344:101::ffff, policy: InService, state: Active, sled_agent_gen: Generation(Generation(1)), repo_depot_port: SqlU16(12348) }]
58662025-06-03T00:56:11.262ZDEBGnexus (service-firewall-rule-progator): sending firewall rules to sled agents background_task = service_firewall_rule_propagation
58672025-06-03T00:56:11.306ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
58682025-06-03T00:56:11.306ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
58692025-06-03T00:56:11.309ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
58702025-06-03T00:56:11.309ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
58712025-06-03T00:56:11.311ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
58722025-06-03T00:56:11.314ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Sled { parent: Fleet, key: afe89ed7-19b7-479b-81eb-05c988ebc66d, lookup_type: ById(afe89ed7-19b7-479b-81eb-05c988ebc66d) } result = Ok(())
58732025-06-03T00:56:11.318ZDEBGnexus (service-firewall-rule-progator): client request SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = service_firewall_rule_propagation body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules
58742025-06-03T00:56:11.319ZDEBGnexus (service-firewall-rule-progator): client response SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = service_firewall_rule_propagation result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules", status: 204, headers: {"x-request-id": "9dea62a5-3f70-4b8e-a595-2786952a06d4", "date": "Tue, 03 Jun 2025 00:56:10 GMT"} })
58752025-06-03T00:56:11.321ZINFOnexus (service-firewall-rule-progator): updated firewall rules on 1 sleds background_task = service_firewall_rule_propagation file = nexus/networking/src/firewall_rules.rs:495
58762025-06-03T00:56:11.321ZDEBGnexus (service-firewall-rule-progator): successfully propagated service firewall rules background_task = service_firewall_rule_propagation elapsed = 816.394909ms
58772025-06-03T00:56:11.321ZDEBGnexus (BackgroundTasks): activation complete background_task = service_firewall_rule_propagation elapsed = 816.410141ms iteration = 2 status = {}
58782025-06-03T00:56:11.346ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "8453b764-bd8b-4fa9-96a1-2e4481c59a3b", "date": "Tue, 03 Jun 2025 00:56:11 GMT"} })
58792025-06-03T00:56:11.346ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae switch = Switch0
58802025-06-03T00:56:11.346ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
58812025-06-03T00:56:11.346ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "01142c14-29d7-406a-bf13-a6b21bf556e2", "content-length": "682", "date": "Tue, 03 Jun 2025 00:56:10 GMT"} })
58822025-06-03T00:56:11.346ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
58832025-06-03T00:56:11.347ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
58842025-06-03T00:56:11.351ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
58852025-06-03T00:56:11.352ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
58862025-06-03T00:56:11.356ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
58872025-06-03T00:56:11.356ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
58882025-06-03T00:56:11.358ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
58892025-06-03T00:56:11.359ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
58902025-06-03T00:56:11.369ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 864.003038ms iteration = 11 status = {}
58912025-06-03T00:56:15.672ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 418 local_addr = [fd00:1122:3344:101::a]:63576 method = GET remote_addr = [fd00:1122:3344:101::d]:61315 req_id = 138d052e-9c1a-45ec-8526-f8d06fd1af53 response_code = 200 uri = /a2861392-5fd2-462b-84e5-6f36c716263a
58922025-06-03T00:56:20.504ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 32 reason = Timeout
58932025-06-03T00:56:20.504ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 32 reason = Timeout
58942025-06-03T00:56:20.504ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
58952025-06-03T00:56:20.504ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
58962025-06-03T00:56:20.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
58972025-06-03T00:56:20.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
58982025-06-03T00:56:20.507ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
58992025-06-03T00:56:20.509ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
59002025-06-03T00:56:20.509ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
59012025-06-03T00:56:20.509ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
59022025-06-03T00:56:20.512ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
59032025-06-03T00:56:20.512ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
59042025-06-03T00:56:20.513ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
59052025-06-03T00:56:20.516ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 4ec93699-353f-4b9e-ae06-25355fdb069e, lookup_type: ById(4ec93699-353f-4b9e-ae06-25355fdb069e) } result = Ok(())
59062025-06-03T00:56:20.516ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
59072025-06-03T00:56:20.516ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
59082025-06-03T00:56:20.527ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 4ec93699-353f-4b9e-ae06-25355fdb069e original_time_created = 2025-06-03 00:51:07.059326 UTC target_id = 4ec93699-353f-4b9e-ae06-25355fdb069e time_created = 2025-06-03 00:51:07.059326 UTC
59092025-06-03T00:56:20.527ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 22.754997ms iteration = 32 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"4ec93699-353f-4b9e-ae06-25355fdb069e","time_created":"2025-06-03T00:51:07.059326Z"}
59102025-06-03T00:56:20.536ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
59112025-06-03T00:56:20.536ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
59122025-06-03T00:56:20.572ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 67.3139ms iteration = 32 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
59132025-06-03T00:56:25.672ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 415 local_addr = [fd00:1122:3344:101::a]:63576 method = GET remote_addr = [fd00:1122:3344:101::d]:61315 req_id = 32f5f1a3-1b3b-46e5-a0cd-f42abd9855b6 response_code = 200 uri = /a2861392-5fd2-462b-84e5-6f36c716263a
59142025-06-03T00:56:30.505ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 33 reason = Timeout
59152025-06-03T00:56:30.505ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
59162025-06-03T00:56:30.505ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 33 reason = Timeout
59172025-06-03T00:56:30.505ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
59182025-06-03T00:56:30.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
59192025-06-03T00:56:30.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
59202025-06-03T00:56:30.507ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
59212025-06-03T00:56:30.508ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
59222025-06-03T00:56:30.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
59232025-06-03T00:56:30.510ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
59242025-06-03T00:56:30.510ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
59252025-06-03T00:56:30.510ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
59262025-06-03T00:56:30.512ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
59272025-06-03T00:56:30.512ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
59282025-06-03T00:56:30.514ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
59292025-06-03T00:56:30.516ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 4ec93699-353f-4b9e-ae06-25355fdb069e, lookup_type: ById(4ec93699-353f-4b9e-ae06-25355fdb069e) } result = Ok(())
59302025-06-03T00:56:30.516ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
59312025-06-03T00:56:30.516ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
59322025-06-03T00:56:30.527ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 4ec93699-353f-4b9e-ae06-25355fdb069e original_time_created = 2025-06-03 00:51:07.059326 UTC target_id = 4ec93699-353f-4b9e-ae06-25355fdb069e time_created = 2025-06-03 00:51:07.059326 UTC
59332025-06-03T00:56:30.527ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 21.940412ms iteration = 33 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"4ec93699-353f-4b9e-ae06-25355fdb069e","time_created":"2025-06-03T00:51:07.059326Z"}
59342025-06-03T00:56:30.541ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 36.919947ms iteration = 33 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
59352025-06-03T00:56:35.672ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 415 local_addr = [fd00:1122:3344:101::a]:63576 method = GET remote_addr = [fd00:1122:3344:101::d]:61315 req_id = 0a383d27-2224-44e3-af4f-d9c6afee490b response_code = 200 uri = /a2861392-5fd2-462b-84e5-6f36c716263a
59362025-06-03T00:56:40.504ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 34 reason = Timeout
59372025-06-03T00:56:40.504ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
59382025-06-03T00:56:40.504ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 12 reason = Timeout
59392025-06-03T00:56:40.504ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 12 reason = Timeout
59402025-06-03T00:56:40.504ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
59412025-06-03T00:56:40.504ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
59422025-06-03T00:56:40.504ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 12 reason = Timeout
59432025-06-03T00:56:40.504ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 12 reason = Timeout
59442025-06-03T00:56:40.504ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
59452025-06-03T00:56:40.504ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
59462025-06-03T00:56:40.504ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 12 reason = Timeout
59472025-06-03T00:56:40.504ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
59482025-06-03T00:56:40.504ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
59492025-06-03T00:56:40.504ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 12 reason = Timeout
59502025-06-03T00:56:40.505ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 34 reason = Timeout
59512025-06-03T00:56:40.505ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
59522025-06-03T00:56:40.505ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 12 reason = Timeout
59532025-06-03T00:56:40.505ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
59542025-06-03T00:56:40.505ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
59552025-06-03T00:56:40.505ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 12 reason = Timeout
59562025-06-03T00:56:40.505ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
59572025-06-03T00:56:40.505ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 12 reason = Timeout
59582025-06-03T00:56:40.505ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
59592025-06-03T00:56:40.505ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 12 reason = Timeout
59602025-06-03T00:56:40.505ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
59612025-06-03T00:56:40.505ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 12 reason = Timeout
59622025-06-03T00:56:40.505ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
59632025-06-03T00:56:40.505ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 12 reason = Timeout
59642025-06-03T00:56:40.505ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
59652025-06-03T00:56:40.505ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 12 reason = Timeout
59662025-06-03T00:56:40.505ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 12 reason = Timeout
59672025-06-03T00:56:40.505ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
59682025-06-03T00:56:40.505ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
59692025-06-03T00:56:40.505ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 12 reason = Timeout
59702025-06-03T00:56:40.505ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
59712025-06-03T00:56:40.505ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
59722025-06-03T00:56:40.505ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 12 reason = Timeout
59732025-06-03T00:56:40.505ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 13 reason = Timeout
59742025-06-03T00:56:40.505ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
59752025-06-03T00:56:40.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
59762025-06-03T00:56:40.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
59772025-06-03T00:56:40.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
59782025-06-03T00:56:40.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
59792025-06-03T00:56:40.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
59802025-06-03T00:56:40.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
59812025-06-03T00:56:40.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
59822025-06-03T00:56:40.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
59832025-06-03T00:56:40.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
59842025-06-03T00:56:40.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
59852025-06-03T00:56:40.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
59862025-06-03T00:56:40.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
59872025-06-03T00:56:40.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
59882025-06-03T00:56:40.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
59892025-06-03T00:56:40.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
59902025-06-03T00:56:40.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
59912025-06-03T00:56:40.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
59922025-06-03T00:56:40.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
59932025-06-03T00:56:40.507ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
59942025-06-03T00:56:40.508ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
59952025-06-03T00:56:40.509ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
59962025-06-03T00:56:40.509ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
59972025-06-03T00:56:40.509ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
59982025-06-03T00:56:40.509ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 4.326094ms iteration = 12 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
59992025-06-03T00:56:40.510ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
60002025-06-03T00:56:40.510ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
60012025-06-03T00:56:40.510ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
60022025-06-03T00:56:40.512ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
60032025-06-03T00:56:40.514ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
60042025-06-03T00:56:40.514ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
60052025-06-03T00:56:40.515ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
60062025-06-03T00:56:40.518ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-afe89ed7-19b7-479b-81eb-05c988ebc66d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-afe89ed7-19b7-479b-81eb-05c988ebc66d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 2727, tv_nsec: 743930050 } })
60072025-06-03T00:56:40.518ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1179
60082025-06-03T00:56:40.519ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-afe89ed7-19b7-479b-81eb-05c988ebc66d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-afe89ed7-19b7-479b-81eb-05c988ebc66d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 2727, tv_nsec: 745150048 } })
60092025-06-03T00:56:40.520ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1179
60102025-06-03T00:56:40.525ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
60112025-06-03T00:56:40.530ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
60122025-06-03T00:56:40.530ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
60132025-06-03T00:56:40.530ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
60142025-06-03T00:56:40.530ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
60152025-06-03T00:56:40.530ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-afe89ed7-19b7-479b-81eb-05c988ebc66d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-afe89ed7-19b7-479b-81eb-05c988ebc66d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 2727, tv_nsec: 756000825 } })
60162025-06-03T00:56:40.531ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1179 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae
60172025-06-03T00:56:40.532ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
60182025-06-03T00:56:40.533ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
60192025-06-03T00:56:40.533ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
60202025-06-03T00:56:40.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
60212025-06-03T00:56:40.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
60222025-06-03T00:56:40.565ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
60232025-06-03T00:56:40.565ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
60242025-06-03T00:56:40.565ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 60.958243ms iteration = 12 status = {"errors":[],"garbage_collect_requested":[]}
60252025-06-03T00:56:40.566ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
60262025-06-03T00:56:40.567ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
60272025-06-03T00:56:40.567ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
60282025-06-03T00:56:40.567ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
60292025-06-03T00:56:40.567ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
60302025-06-03T00:56:40.569ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 64.694129ms iteration = 12 status = {"errors":[],"requests_created_ok":[]}
60312025-06-03T00:56:40.569ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
60322025-06-03T00:56:40.569ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
60332025-06-03T00:56:40.569ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
60342025-06-03T00:56:40.570ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
60352025-06-03T00:56:40.570ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
60362025-06-03T00:56:40.570ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
60372025-06-03T00:56:40.570ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
60382025-06-03T00:56:40.573ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
60392025-06-03T00:56:40.573ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
60402025-06-03T00:56:40.574ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 69.197619ms iteration = 12 status = {"errors":[],"finish_invoked_ok":[]}
60412025-06-03T00:56:40.575ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
60422025-06-03T00:56:40.575ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
60432025-06-03T00:56:40.575ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
60442025-06-03T00:56:40.575ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
60452025-06-03T00:56:40.576ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
60462025-06-03T00:56:40.577ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
60472025-06-03T00:56:40.577ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
60482025-06-03T00:56:40.578ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
60492025-06-03T00:56:40.579ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
60502025-06-03T00:56:40.579ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
60512025-06-03T00:56:40.579ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 4ec93699-353f-4b9e-ae06-25355fdb069e, lookup_type: ById(4ec93699-353f-4b9e-ae06-25355fdb069e) } result = Ok(())
60522025-06-03T00:56:40.579ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
60532025-06-03T00:56:40.579ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
60542025-06-03T00:56:40.579ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
60552025-06-03T00:56:40.579ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
60562025-06-03T00:56:40.580ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
60572025-06-03T00:56:40.580ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
60582025-06-03T00:56:40.580ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
60592025-06-03T00:56:40.582ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
60602025-06-03T00:56:40.582ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
60612025-06-03T00:56:40.582ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
60622025-06-03T00:56:40.582ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
60632025-06-03T00:56:40.583ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
60642025-06-03T00:56:40.583ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
60652025-06-03T00:56:40.584ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
60662025-06-03T00:56:40.584ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
60672025-06-03T00:56:40.585ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
60682025-06-03T00:56:40.585ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
60692025-06-03T00:56:40.586ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
60702025-06-03T00:56:40.586ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
60712025-06-03T00:56:40.588ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
60722025-06-03T00:56:40.588ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
60732025-06-03T00:56:40.588ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
60742025-06-03T00:56:40.589ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
60752025-06-03T00:56:40.589ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
60762025-06-03T00:56:40.589ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
60772025-06-03T00:56:40.589ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
60782025-06-03T00:56:40.589ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
60792025-06-03T00:56:40.589ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
60802025-06-03T00:56:40.590ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: afe89ed7-19b7-479b-81eb-05c988ebc66d, lookup_type: ById(afe89ed7-19b7-479b-81eb-05c988ebc66d) } result = Ok(())
60812025-06-03T00:56:40.590ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
60822025-06-03T00:56:40.590ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
60832025-06-03T00:56:40.590ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
60842025-06-03T00:56:40.590ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
60852025-06-03T00:56:40.596ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 91.400518ms iteration = 12 status = 0
60862025-06-03T00:56:40.596ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
60872025-06-03T00:56:40.598ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
60882025-06-03T00:56:40.598ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
60892025-06-03T00:56:40.598ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 4ec93699-353f-4b9e-ae06-25355fdb069e original_time_created = 2025-06-03 00:51:07.059326 UTC target_id = 4ec93699-353f-4b9e-ae06-25355fdb069e time_created = 2025-06-03 00:51:07.059326 UTC
60902025-06-03T00:56:40.598ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 93.578405ms iteration = 34 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"4ec93699-353f-4b9e-ae06-25355fdb069e","time_created":"2025-06-03T00:51:07.059326Z"}
60912025-06-03T00:56:40.598ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
60922025-06-03T00:56:40.606ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1187 zone_address = fd00:1122:3344:101::2
60932025-06-03T00:56:40.606ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
60942025-06-03T00:56:40.606ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "3c0ae8b1-a0e0-4518-8c56-ffe7e1400372", "content-length": "26", "date": "Tue, 03 Jun 2025 00:56:40 GMT"} })
60952025-06-03T00:56:40.606ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1190 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
60962025-06-03T00:56:40.607ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1224 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
60972025-06-03T00:56:40.618ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 113.635709ms iteration = 34 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
60982025-06-03T00:56:40.621ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
60992025-06-03T00:56:40.621ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
61002025-06-03T00:56:40.621ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 116.095165ms iteration = 12 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
61012025-06-03T00:56:40.628ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 123.574727ms iteration = 12 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
61022025-06-03T00:56:40.630ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 125.102785ms iteration = 12 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
61032025-06-03T00:56:40.630ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 125.569999ms iteration = 12 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
61042025-06-03T00:56:40.648ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:372
61052025-06-03T00:56:40.648ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 143.603349ms iteration = 12 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
61062025-06-03T00:56:40.649ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:547 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
61072025-06-03T00:56:40.649ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 144.152587ms iteration = 12 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
61082025-06-03T00:56:40.650ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
61092025-06-03T00:56:40.652ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
61102025-06-03T00:56:40.652ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
61112025-06-03T00:56:40.652ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
61122025-06-03T00:56:40.702ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 197.447702ms iteration = 13 status = {"physical_disks_added":0}
61132025-06-03T00:56:40.708ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1187 zone_address = fd00:1122:3344:101::2
61142025-06-03T00:56:40.708ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
61152025-06-03T00:56:40.708ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "315ad54f-9bd8-4ee9-80da-9967905d23be", "content-length": "26", "date": "Tue, 03 Jun 2025 00:56:40 GMT"} })
61162025-06-03T00:56:40.708ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1190 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
61172025-06-03T00:56:40.710ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1224 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
61182025-06-03T00:56:40.715ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
61192025-06-03T00:56:40.715ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
61202025-06-03T00:56:40.722ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1187 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae zone_address = fd00:1122:3344:101::2
61212025-06-03T00:56:40.722ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
61222025-06-03T00:56:40.722ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "5edb3512-6292-4ecd-a33b-644b91c0b355", "content-length": "26", "date": "Tue, 03 Jun 2025 00:56:40 GMT"} })
61232025-06-03T00:56:40.722ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1190 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
61242025-06-03T00:56:40.724ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1224 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae
61252025-06-03T00:56:40.726ZINFOnexus (BackgroundTasks): VPC route manager sled afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
61262025-06-03T00:56:40.726ZDEBGnexus (BackgroundTasks): client request SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
61272025-06-03T00:56:40.728ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 223.958094ms iteration = 12 status = null
61282025-06-03T00:56:40.780ZDEBGnexus (BackgroundTasks): client response SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "cee24578-0ddd-49f4-af37-2bafc8f31528", "content-length": "372", "date": "Tue, 03 Jun 2025 00:56:40 GMT"} })
61292025-06-03T00:56:40.780ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
61302025-06-03T00:56:40.781ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
61312025-06-03T00:56:40.814ZDEBGnexus (BackgroundTasks): client request SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
61322025-06-03T00:56:40.815ZDEBGnexus (BackgroundTasks): client response SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "c59a455d-ff4e-4abb-aa4e-0b3a377f949a", "content-length": "691", "date": "Tue, 03 Jun 2025 00:56:40 GMT"} })
61332025-06-03T00:56:40.815ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
61342025-06-03T00:56:40.815ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
61352025-06-03T00:56:40.815ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
61362025-06-03T00:56:40.816ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 311.448831ms iteration = 12 status = {}
61372025-06-03T00:56:40.827ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
61382025-06-03T00:56:40.827ZDEBGnexus (BackgroundTasks): client request SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
61392025-06-03T00:56:40.827ZDEBGnexus (BackgroundTasks): client response SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "2f3d9f30-34c5-46b9-937c-ef0270061748", "date": "Tue, 03 Jun 2025 00:56:40 GMT"} })
61402025-06-03T00:56:40.827ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
61412025-06-03T00:56:40.828ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
61422025-06-03T00:56:40.830ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
61432025-06-03T00:56:40.831ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
61442025-06-03T00:56:40.833ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
61452025-06-03T00:56:40.833ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
61462025-06-03T00:56:40.836ZINFOnexus (BackgroundTasks): VPC route manager sled afe89ed7-19b7-479b-81eb-05c988ebc66d push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
61472025-06-03T00:56:40.836ZDEBGnexus (BackgroundTasks): client request SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
61482025-06-03T00:56:40.838ZDEBGnexus (BackgroundTasks): client response SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "7b183808-4976-4f32-a20b-f3222080eb57", "date": "Tue, 03 Jun 2025 00:56:40 GMT"} })
61492025-06-03T00:56:40.839ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 334.92247ms iteration = 12 status = {}
61502025-06-03T00:56:40.861ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 356.239662ms iteration = 12 status = {}
61512025-06-03T00:56:40.922ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
61522025-06-03T00:56:40.922ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
61532025-06-03T00:56:40.924ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: ec5e33f9-c697-46e2-a93e-a745336bf9dc, rack_id: d6831ba0-55f0-4191-8dc2-ea15e55ab0ae, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(9378a284-0ee9-48e2-80a0-3bcaadb1c4e2) } rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae switch_location = Switch0
61542025-06-03T00:56:40.924ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
61552025-06-03T00:56:40.924ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
61562025-06-03T00:56:40.936ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1832 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
61572025-06-03T00:56:40.936ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1859 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 9378a284-0ee9-48e2-80a0-3bcaadb1c4e2
61582025-06-03T00:56:40.937ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
61592025-06-03T00:56:40.937ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
61602025-06-03T00:56:40.937ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
61612025-06-03T00:56:40.937ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
61622025-06-03T00:56:40.937ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
61632025-06-03T00:56:40.937ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2006 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
61642025-06-03T00:56:40.937ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:408 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
61652025-06-03T00:56:40.937ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2042 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
61662025-06-03T00:56:40.937ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae
61672025-06-03T00:56:40.937ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
61682025-06-03T00:56:40.937ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
61692025-06-03T00:56:40.940ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae switch_location = Switch0
61702025-06-03T00:56:40.940ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
61712025-06-03T00:56:41.033ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "d10701c4-27f1-4bcc-956d-f7b058496c42", "date": "Tue, 03 Jun 2025 00:56:40 GMT"} })
61722025-06-03T00:56:41.033ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae switch = Switch0
61732025-06-03T00:56:41.033ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
61742025-06-03T00:56:41.033ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "259310ea-b8f2-4181-888c-68d06d71efc1", "content-length": "682", "date": "Tue, 03 Jun 2025 00:56:40 GMT"} })
61752025-06-03T00:56:41.033ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
61762025-06-03T00:56:41.033ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
61772025-06-03T00:56:41.037ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
61782025-06-03T00:56:41.037ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
61792025-06-03T00:56:41.039ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
61802025-06-03T00:56:41.040ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
61812025-06-03T00:56:41.041ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
61822025-06-03T00:56:41.042ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
61832025-06-03T00:56:41.049ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 544.475947ms iteration = 12 status = {}
61842025-06-03T00:56:45.673ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 435 local_addr = [fd00:1122:3344:101::a]:63576 method = GET remote_addr = [fd00:1122:3344:101::d]:61315 req_id = ffe02dff-7287-4a21-b191-66f585f5364b response_code = 200 uri = /a2861392-5fd2-462b-84e5-6f36c716263a
61852025-06-03T00:56:50.505ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 35 reason = Timeout
61862025-06-03T00:56:50.505ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
61872025-06-03T00:56:50.505ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 35 reason = Timeout
61882025-06-03T00:56:50.505ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
61892025-06-03T00:56:50.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
61902025-06-03T00:56:50.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
61912025-06-03T00:56:50.507ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
61922025-06-03T00:56:50.510ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
61932025-06-03T00:56:50.510ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
61942025-06-03T00:56:50.510ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
61952025-06-03T00:56:50.519ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
61962025-06-03T00:56:50.520ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
61972025-06-03T00:56:50.521ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
61982025-06-03T00:56:50.524ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 4ec93699-353f-4b9e-ae06-25355fdb069e, lookup_type: ById(4ec93699-353f-4b9e-ae06-25355fdb069e) } result = Ok(())
61992025-06-03T00:56:50.524ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
62002025-06-03T00:56:50.524ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
62012025-06-03T00:56:50.533ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 4ec93699-353f-4b9e-ae06-25355fdb069e original_time_created = 2025-06-03 00:51:07.059326 UTC target_id = 4ec93699-353f-4b9e-ae06-25355fdb069e time_created = 2025-06-03 00:51:07.059326 UTC
62022025-06-03T00:56:50.533ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 28.836875ms iteration = 35 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"4ec93699-353f-4b9e-ae06-25355fdb069e","time_created":"2025-06-03T00:51:07.059326Z"}
62032025-06-03T00:56:50.535ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
62042025-06-03T00:56:50.535ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
62052025-06-03T00:56:50.536ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 31.099756ms iteration = 35 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
62062025-06-03T00:56:55.672ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 284 local_addr = [fd00:1122:3344:101::a]:63576 method = GET remote_addr = [fd00:1122:3344:101::d]:61315 req_id = ca3701ca-cec3-4453-b809-cd54058c2d83 response_code = 200 uri = /a2861392-5fd2-462b-84e5-6f36c716263a
62072025-06-03T00:57:00.504ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 36 reason = Timeout
62082025-06-03T00:57:00.504ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 36 reason = Timeout
62092025-06-03T00:57:00.504ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
62102025-06-03T00:57:00.504ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
62112025-06-03T00:57:00.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
62122025-06-03T00:57:00.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
62132025-06-03T00:57:00.506ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
62142025-06-03T00:57:00.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
62152025-06-03T00:57:00.506ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62162025-06-03T00:57:00.507ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.326533ms iteration = 36 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
62172025-06-03T00:57:00.508ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
62182025-06-03T00:57:00.508ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
62192025-06-03T00:57:00.509ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
62202025-06-03T00:57:00.510ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
62212025-06-03T00:57:00.510ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
62222025-06-03T00:57:00.542ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62232025-06-03T00:57:00.545ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 4ec93699-353f-4b9e-ae06-25355fdb069e, lookup_type: ById(4ec93699-353f-4b9e-ae06-25355fdb069e) } result = Ok(())
62242025-06-03T00:57:00.545ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
62252025-06-03T00:57:00.545ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
62262025-06-03T00:57:00.555ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 4ec93699-353f-4b9e-ae06-25355fdb069e original_time_created = 2025-06-03 00:51:07.059326 UTC target_id = 4ec93699-353f-4b9e-ae06-25355fdb069e time_created = 2025-06-03 00:51:07.059326 UTC
62272025-06-03T00:57:00.555ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 50.8838ms iteration = 36 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"4ec93699-353f-4b9e-ae06-25355fdb069e","time_created":"2025-06-03T00:51:07.059326Z"}
62282025-06-03T00:57:05.672ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 303 local_addr = [fd00:1122:3344:101::a]:63576 method = GET remote_addr = [fd00:1122:3344:101::d]:61315 req_id = 405ae3d8-21cb-4d0b-b2f6-3bfb84efbd3e response_code = 200 uri = /a2861392-5fd2-462b-84e5-6f36c716263a
62292025-06-03T00:57:10.503ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 7 reason = Timeout
62302025-06-03T00:57:10.503ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
62312025-06-03T00:57:10.503ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
62322025-06-03T00:57:10.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
62332025-06-03T00:57:10.504ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 37 reason = Timeout
62342025-06-03T00:57:10.504ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 13 reason = Timeout
62352025-06-03T00:57:10.504ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
62362025-06-03T00:57:10.504ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
62372025-06-03T00:57:10.504ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 13 reason = Timeout
62382025-06-03T00:57:10.504ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 13 reason = Timeout
62392025-06-03T00:57:10.504ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
62402025-06-03T00:57:10.504ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 13 reason = Timeout
62412025-06-03T00:57:10.504ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
62422025-06-03T00:57:10.504ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
62432025-06-03T00:57:10.504ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 13 reason = Timeout
62442025-06-03T00:57:10.504ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
62452025-06-03T00:57:10.504ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
62462025-06-03T00:57:10.504ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 13 reason = Timeout
62472025-06-03T00:57:10.504ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
62482025-06-03T00:57:10.504ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 37 reason = Timeout
62492025-06-03T00:57:10.504ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
62502025-06-03T00:57:10.504ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 13 reason = Timeout
62512025-06-03T00:57:10.504ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
62522025-06-03T00:57:10.504ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
62532025-06-03T00:57:10.504ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 13 reason = Timeout
62542025-06-03T00:57:10.504ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
62552025-06-03T00:57:10.504ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 13 reason = Timeout
62562025-06-03T00:57:10.504ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 13 reason = Timeout
62572025-06-03T00:57:10.504ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
62582025-06-03T00:57:10.504ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
62592025-06-03T00:57:10.504ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 7 reason = Timeout
62602025-06-03T00:57:10.504ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
62612025-06-03T00:57:10.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
62622025-06-03T00:57:10.504ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 13 reason = Timeout
62632025-06-03T00:57:10.504ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 9 reason = Timeout
62642025-06-03T00:57:10.504ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
62652025-06-03T00:57:10.504ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 13 reason = Timeout
62662025-06-03T00:57:10.504ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
62672025-06-03T00:57:10.504ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 7 reason = Timeout
62682025-06-03T00:57:10.504ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 7 reason = Timeout
62692025-06-03T00:57:10.504ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-06-03 00:47:10.504729005 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
62702025-06-03T00:57:10.504ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
62712025-06-03T00:57:10.504ZDEBGnexus (BackgroundTasks): activating background_task = webhook_deliverator iteration = 7 reason = Timeout
62722025-06-03T00:57:10.504ZDEBGnexus (BackgroundTasks): roles background_task = webhook_deliverator roles = RoleSet { roles: {} }
62732025-06-03T00:57:10.504ZDEBGnexus (BackgroundTasks): activating background_task = alert_dispatcher iteration = 7 reason = Timeout
62742025-06-03T00:57:10.504ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
62752025-06-03T00:57:10.504ZDEBGnexus (BackgroundTasks): activating background_task = crdb_node_id_collector iteration = 4 reason = Timeout
62762025-06-03T00:57:10.504ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 7 reason = Timeout
62772025-06-03T00:57:10.504ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:53
62782025-06-03T00:57:10.504ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
62792025-06-03T00:57:10.504ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
62802025-06-03T00:57:10.505ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 9 reason = Timeout
62812025-06-03T00:57:10.505ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 14 reason = Timeout
62822025-06-03T00:57:10.505ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
62832025-06-03T00:57:10.505ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 13 reason = Timeout
62842025-06-03T00:57:10.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
62852025-06-03T00:57:10.505ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
62862025-06-03T00:57:10.505ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 7 reason = Timeout
62872025-06-03T00:57:10.505ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
62882025-06-03T00:57:10.505ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 7 reason = Timeout
62892025-06-03T00:57:10.505ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
62902025-06-03T00:57:10.505ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
62912025-06-03T00:57:10.505ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 7 reason = Timeout
62922025-06-03T00:57:10.505ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
62932025-06-03T00:57:10.505ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 8 reason = Timeout
62942025-06-03T00:57:10.505ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:85 reason = blueprint disabled target_id = 4ec93699-353f-4b9e-ae06-25355fdb069e
62952025-06-03T00:57:10.505ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 7 reason = Timeout
62962025-06-03T00:57:10.505ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 11.47\u{b5}s iteration = 8 status = {"enabled":false,"target_id":"4ec93699-353f-4b9e-ae06-25355fdb069e"}
62972025-06-03T00:57:10.505ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
62982025-06-03T00:57:10.505ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 7 reason = Timeout
62992025-06-03T00:57:10.505ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 13 reason = Timeout
63002025-06-03T00:57:10.505ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 13 reason = Timeout
63012025-06-03T00:57:10.505ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
63022025-06-03T00:57:10.505ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 13 reason = Timeout
63032025-06-03T00:57:10.505ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
63042025-06-03T00:57:10.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
63052025-06-03T00:57:10.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
63062025-06-03T00:57:10.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
63072025-06-03T00:57:10.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
63082025-06-03T00:57:10.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
63092025-06-03T00:57:10.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
63102025-06-03T00:57:10.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
63112025-06-03T00:57:10.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
63122025-06-03T00:57:10.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
63132025-06-03T00:57:10.505ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
63142025-06-03T00:57:10.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
63152025-06-03T00:57:10.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
63162025-06-03T00:57:10.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
63172025-06-03T00:57:10.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
63182025-06-03T00:57:10.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
63192025-06-03T00:57:10.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
63202025-06-03T00:57:10.505ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
63212025-06-03T00:57:10.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
63222025-06-03T00:57:10.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = alert_dispatcher resource = Database result = Ok(())
63232025-06-03T00:57:10.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
63242025-06-03T00:57:10.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
63252025-06-03T00:57:10.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
63262025-06-03T00:57:10.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
63272025-06-03T00:57:10.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
63282025-06-03T00:57:10.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
63292025-06-03T00:57:10.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = webhook_deliverator resource = Database result = Ok(())
63302025-06-03T00:57:10.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
63312025-06-03T00:57:10.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
63322025-06-03T00:57:10.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
63332025-06-03T00:57:10.505ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("9feb16c9-ff60-43e8-917b-965124f46e0d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("aa124775-31de-4648-981a-4af500268b66.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("aa124775-31de-4648-981a-4af500268b66.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::9))) }], valid_until: Instant { tv_sec: 2757, tv_nsec: 731370950 } })
63342025-06-03T00:57:10.505ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("241573bc-467c-4802-9f4b-f7c82da4e12e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("64fc0a69-d4b8-4f6c-9a07-77a4cfa94962.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("ceb74144-4b6b-4b3f-bc80-eac2d75f3bc9.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("ceb74144-4b6b-4b3f-bc80-eac2d75f3bc9.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:2::1))) }], valid_until: Instant { tv_sec: 2757, tv_nsec: 731378670 } })
63352025-06-03T00:57:10.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
63362025-06-03T00:57:10.505ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
63372025-06-03T00:57:10.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
63382025-06-03T00:57:10.506ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
63392025-06-03T00:57:10.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
63402025-06-03T00:57:10.506ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
63412025-06-03T00:57:10.506ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
63422025-06-03T00:57:10.506ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.355322ms iteration = 7 status = {"addresses":["[fd00:1122:3344:101::8]:5353","[fd00:1122:3344:101::9]:5353"]}
63432025-06-03T00:57:10.506ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } dns_group = internal
63442025-06-03T00:57:10.506ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.743574ms iteration = 7 status = {"addresses":["[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:3::1]:5353","[fd00:1122:3344:2::1]:5353"]}
63452025-06-03T00:57:10.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
63462025-06-03T00:57:10.507ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
63472025-06-03T00:57:10.512ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
63482025-06-03T00:57:10.512ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
63492025-06-03T00:57:10.512ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
63502025-06-03T00:57:10.512ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 7.826173ms iteration = 13 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
63512025-06-03T00:57:10.516ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-afe89ed7-19b7-479b-81eb-05c988ebc66d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-afe89ed7-19b7-479b-81eb-05c988ebc66d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 2757, tv_nsec: 742245938 } })
63522025-06-03T00:57:10.517ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1179
63532025-06-03T00:57:10.536ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-06-03 00:47:10.504729005 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
63542025-06-03T00:57:10.536ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 32.174739ms iteration = 7 status = {"expiration":"2025-06-03T00:47:10.504729005Z","pruned":[]}
63552025-06-03T00:57:10.540ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
63562025-06-03T00:57:10.543ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
63572025-06-03T00:57:10.543ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
63582025-06-03T00:57:10.544ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
63592025-06-03T00:57:10.554ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
63602025-06-03T00:57:10.557ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
63612025-06-03T00:57:10.557ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
63622025-06-03T00:57:10.557ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
63632025-06-03T00:57:10.560ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 55.620488ms iteration = 7 status = {"errors":[],"found_port_ok":[]}
63642025-06-03T00:57:10.561ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
63652025-06-03T00:57:10.561ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
63662025-06-03T00:57:10.589ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 85.244807ms iteration = 13 status = {"errors":[],"garbage_collect_requested":[]}
63672025-06-03T00:57:10.591ZINFOnexus (BackgroundTasks): all webhook delivery tasks completed background_task = webhook_deliverator file = nexus/src/app/background/tasks/webhook_deliverator.rs:202 num_receivers = 0
63682025-06-03T00:57:10.591ZDEBGnexus (BackgroundTasks): activation complete background_task = webhook_deliverator elapsed = 87.078024ms iteration = 7 status = {"by_rx":{},"error":null}
63692025-06-03T00:57:10.594ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
63702025-06-03T00:57:10.597ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
63712025-06-03T00:57:10.597ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
63722025-06-03T00:57:10.597ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
63732025-06-03T00:57:10.598ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
63742025-06-03T00:57:10.600ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
63752025-06-03T00:57:10.600ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
63762025-06-03T00:57:10.600ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
63772025-06-03T00:57:10.603ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
63782025-06-03T00:57:10.603ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
63792025-06-03T00:57:10.604ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-afe89ed7-19b7-479b-81eb-05c988ebc66d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-afe89ed7-19b7-479b-81eb-05c988ebc66d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 2757, tv_nsec: 829645142 } })
63802025-06-03T00:57:10.604ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1179
63812025-06-03T00:57:10.605ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
63822025-06-03T00:57:10.606ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
63832025-06-03T00:57:10.607ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
63842025-06-03T00:57:10.609ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
63852025-06-03T00:57:10.609ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
63862025-06-03T00:57:10.609ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
63872025-06-03T00:57:10.609ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
63882025-06-03T00:57:10.610ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
63892025-06-03T00:57:10.612ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
63902025-06-03T00:57:10.613ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
63912025-06-03T00:57:10.613ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
63922025-06-03T00:57:10.613ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
63932025-06-03T00:57:10.614ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
63942025-06-03T00:57:10.614ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
63952025-06-03T00:57:10.616ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
63962025-06-03T00:57:10.616ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
63972025-06-03T00:57:10.617ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
63982025-06-03T00:57:10.617ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
63992025-06-03T00:57:10.617ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
64002025-06-03T00:57:10.617ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
64012025-06-03T00:57:10.617ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
64022025-06-03T00:57:10.618ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
64032025-06-03T00:57:10.618ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
64042025-06-03T00:57:10.618ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
64052025-06-03T00:57:10.618ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
64062025-06-03T00:57:10.618ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
64072025-06-03T00:57:10.618ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
64082025-06-03T00:57:10.621ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
64092025-06-03T00:57:10.621ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
64102025-06-03T00:57:10.621ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
64112025-06-03T00:57:10.621ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
64122025-06-03T00:57:10.621ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
64132025-06-03T00:57:10.621ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
64142025-06-03T00:57:10.622ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
64152025-06-03T00:57:10.622ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
64162025-06-03T00:57:10.623ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 118.320604ms iteration = 13 status = {"errors":[],"finish_invoked_ok":[]}
64172025-06-03T00:57:10.623ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
64182025-06-03T00:57:10.625ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
64192025-06-03T00:57:10.625ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
64202025-06-03T00:57:10.625ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
64212025-06-03T00:57:10.626ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
64222025-06-03T00:57:10.626ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
64232025-06-03T00:57:10.626ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
64242025-06-03T00:57:10.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
64252025-06-03T00:57:10.627ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
64262025-06-03T00:57:10.627ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1187 zone_address = fd00:1122:3344:101::2
64272025-06-03T00:57:10.627ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
64282025-06-03T00:57:10.627ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
64292025-06-03T00:57:10.627ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "cdaf6bd1-b937-4990-9d73-211c2923a0df", "content-length": "26", "date": "Tue, 03 Jun 2025 00:57:10 GMT"} })
64302025-06-03T00:57:10.627ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1190 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
64312025-06-03T00:57:10.627ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
64322025-06-03T00:57:10.627ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
64332025-06-03T00:57:10.627ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:167 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
64342025-06-03T00:57:10.627ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 123.214955ms iteration = 7 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
64352025-06-03T00:57:10.627ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
64362025-06-03T00:57:10.628ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
64372025-06-03T00:57:10.628ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
64382025-06-03T00:57:10.628ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-afe89ed7-19b7-479b-81eb-05c988ebc66d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-afe89ed7-19b7-479b-81eb-05c988ebc66d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 2757, tv_nsec: 854175153 } })
64392025-06-03T00:57:10.629ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1179 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae
64402025-06-03T00:57:10.629ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
64412025-06-03T00:57:10.629ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1224 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
64422025-06-03T00:57:10.630ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = alert_dispatcher resource = Database result = Ok(())
64432025-06-03T00:57:10.630ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
64442025-06-03T00:57:10.631ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
64452025-06-03T00:57:10.631ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
64462025-06-03T00:57:10.631ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
64472025-06-03T00:57:10.631ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
64482025-06-03T00:57:10.633ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
64492025-06-03T00:57:10.633ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
64502025-06-03T00:57:10.633ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
64512025-06-03T00:57:10.635ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
64522025-06-03T00:57:10.635ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
64532025-06-03T00:57:10.638ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
64542025-06-03T00:57:10.642ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
64552025-06-03T00:57:10.642ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
64562025-06-03T00:57:10.642ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
64572025-06-03T00:57:10.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
64582025-06-03T00:57:10.643ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
64592025-06-03T00:57:10.645ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
64602025-06-03T00:57:10.645ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
64612025-06-03T00:57:10.645ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
64622025-06-03T00:57:10.647ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 143.450345ms iteration = 37 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
64632025-06-03T00:57:10.648ZDEBGnexus (BackgroundTasks): activation complete background_task = crdb_node_id_collector elapsed = 144.114305ms iteration = 4 status = {"nsuccess":5}
64642025-06-03T00:57:10.649ZDEBGnexus (BackgroundTasks): activation complete background_task = alert_dispatcher elapsed = 144.786406ms iteration = 7 status = {"dispatched":[],"errors":[],"glob_version":"144.0.0","globs_reprocessed":{},"no_receivers":[]}
64652025-06-03T00:57:10.661ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:1025 local_addr = [fd00:1122:3344:101::a]:12221 remote_addr = [fd00:1122:3344:101::d]:50103
64662025-06-03T00:57:10.661ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:50103 req_id = 54a51b08-8787-4e99-b91d-b1c7a72569bd roles = RoleSet { roles: {} } uri = /metrics/collectors/292a19e1-4305-4e49-b11e-08ac74714bf3/producers?sort_by=id_ascending
64672025-06-03T00:57:10.662ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:50103 req_id = 54a51b08-8787-4e99-b91d-b1c7a72569bd resource = Database result = Ok(()) uri = /metrics/collectors/292a19e1-4305-4e49-b11e-08ac74714bf3/producers?sort_by=id_ascending
64682025-06-03T00:57:10.666ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 161.411912ms iteration = 13 status = {"errors":[],"requests_created_ok":[]}
64692025-06-03T00:57:10.682ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 21441 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:50103 req_id = 54a51b08-8787-4e99-b91d-b1c7a72569bd response_code = 200 uri = /metrics/collectors/292a19e1-4305-4e49-b11e-08ac74714bf3/producers?sort_by=id_ascending
64702025-06-03T00:57:10.683ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:50103 req_id = c2f9e92c-ee43-4243-a51c-4f27d721bff6 roles = RoleSet { roles: {} } uri = /metrics/collectors/292a19e1-4305-4e49-b11e-08ac74714bf3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmZTEyZmZjMS1mMmFjLTRiZjMtOTY3Mi1mYzE4NWY4MmM1OWIifX0%3D
64712025-06-03T00:57:10.683ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:50103 req_id = c2f9e92c-ee43-4243-a51c-4f27d721bff6 resource = Database result = Ok(()) uri = /metrics/collectors/292a19e1-4305-4e49-b11e-08ac74714bf3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmZTEyZmZjMS1mMmFjLTRiZjMtOTY3Mi1mYzE4NWY4MmM1OWIifX0%3D
64722025-06-03T00:57:10.684ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
64732025-06-03T00:57:10.686ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
64742025-06-03T00:57:10.686ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
64752025-06-03T00:57:10.687ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
64762025-06-03T00:57:10.687ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
64772025-06-03T00:57:10.687ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
64782025-06-03T00:57:10.687ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
64792025-06-03T00:57:10.687ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 4129 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:50103 req_id = c2f9e92c-ee43-4243-a51c-4f27d721bff6 response_code = 200 uri = /metrics/collectors/292a19e1-4305-4e49-b11e-08ac74714bf3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmZTEyZmZjMS1mMmFjLTRiZjMtOTY3Mi1mYzE4NWY4MmM1OWIifX0%3D
64802025-06-03T00:57:10.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
64812025-06-03T00:57:10.687ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
64822025-06-03T00:57:10.687ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
64832025-06-03T00:57:10.688ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
64842025-06-03T00:57:10.689ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
64852025-06-03T00:57:10.692ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 187.533883ms iteration = 13 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
64862025-06-03T00:57:10.692ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
64872025-06-03T00:57:10.692ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
64882025-06-03T00:57:10.693ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
64892025-06-03T00:57:10.698ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
64902025-06-03T00:57:10.698ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
64912025-06-03T00:57:10.698ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
64922025-06-03T00:57:10.698ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
64932025-06-03T00:57:10.700ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
64942025-06-03T00:57:10.702ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
64952025-06-03T00:57:10.703ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
64962025-06-03T00:57:10.704ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
64972025-06-03T00:57:10.704ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:137 found = 0 not_ready_to_be_deleted = 0
64982025-06-03T00:57:10.704ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 199.392662ms iteration = 7 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
64992025-06-03T00:57:10.704ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 4ec93699-353f-4b9e-ae06-25355fdb069e, lookup_type: ById(4ec93699-353f-4b9e-ae06-25355fdb069e) } result = Ok(())
65002025-06-03T00:57:10.704ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
65012025-06-03T00:57:10.705ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
65022025-06-03T00:57:10.710ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
65032025-06-03T00:57:10.710ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
65042025-06-03T00:57:10.715ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 211.173777ms iteration = 13 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
65052025-06-03T00:57:10.719ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
65062025-06-03T00:57:10.719ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
65072025-06-03T00:57:10.719ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
65082025-06-03T00:57:10.719ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
65092025-06-03T00:57:10.719ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
65102025-06-03T00:57:10.719ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:547 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
65112025-06-03T00:57:10.719ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 215.048968ms iteration = 13 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
65122025-06-03T00:57:10.726ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
65132025-06-03T00:57:10.726ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
65142025-06-03T00:57:10.738ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1187 zone_address = fd00:1122:3344:101::2
65152025-06-03T00:57:10.738ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
65162025-06-03T00:57:10.738ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "31dc0a2e-66a1-4a8e-8f97-1049885c739d", "content-length": "26", "date": "Tue, 03 Jun 2025 00:57:10 GMT"} })
65172025-06-03T00:57:10.738ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1190 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
65182025-06-03T00:57:10.741ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1224 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
65192025-06-03T00:57:10.747ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
65202025-06-03T00:57:10.749ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
65212025-06-03T00:57:10.749ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
65222025-06-03T00:57:10.750ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
65232025-06-03T00:57:10.750ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
65242025-06-03T00:57:10.750ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
65252025-06-03T00:57:10.751ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
65262025-06-03T00:57:10.752ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:372
65272025-06-03T00:57:10.752ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 248.432384ms iteration = 13 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
65282025-06-03T00:57:10.753ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
65292025-06-03T00:57:10.754ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
65302025-06-03T00:57:10.782ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 277.809406ms iteration = 13 status = {}
65312025-06-03T00:57:10.797ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1187 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae zone_address = fd00:1122:3344:101::2
65322025-06-03T00:57:10.797ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
65332025-06-03T00:57:10.797ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "bb12f383-6cc4-4f79-a888-e5df1cb0dbe2", "content-length": "26", "date": "Tue, 03 Jun 2025 00:57:10 GMT"} })
65342025-06-03T00:57:10.797ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1190 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
65352025-06-03T00:57:10.799ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1224 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae
65362025-06-03T00:57:10.814ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 309.878232ms iteration = 14 status = {"physical_disks_added":0}
65372025-06-03T00:57:10.907ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
65382025-06-03T00:57:10.907ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "2f5720f8-f0bf-497d-8801-4a7fb5a39e99", "date": "Tue, 03 Jun 2025 00:57:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
65392025-06-03T00:57:10.907ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
65402025-06-03T00:57:10.909ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "9304b42b-1045-494c-b143-04ef4de7993a", "date": "Tue, 03 Jun 2025 00:57:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
65412025-06-03T00:57:10.909ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
65422025-06-03T00:57:10.911ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "6a3d728a-19d4-49d5-8d58-eb0eca45e1b9", "date": "Tue, 03 Jun 2025 00:57:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
65432025-06-03T00:57:10.911ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
65442025-06-03T00:57:10.913ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
65452025-06-03T00:57:10.913ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 408.821795ms iteration = 9 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
65462025-06-03T00:57:10.979ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
65472025-06-03T00:57:10.979ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "d7dcf0ad-e5a0-4296-ac4e-d68916d5f6a3", "date": "Tue, 03 Jun 2025 00:57:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
65482025-06-03T00:57:10.979ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
65492025-06-03T00:57:10.981ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "fffd5bfd-8dd8-4653-9101-255b08490253", "date": "Tue, 03 Jun 2025 00:57:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
65502025-06-03T00:57:10.981ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
65512025-06-03T00:57:10.982ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
65522025-06-03T00:57:10.982ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 477.984812ms iteration = 9 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
65532025-06-03T00:57:11.012ZINFOnexus (BackgroundTasks): VPC route manager sled afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
65542025-06-03T00:57:11.012ZDEBGnexus (BackgroundTasks): client request SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
65552025-06-03T00:57:11.018ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
65562025-06-03T00:57:11.018ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
65572025-06-03T00:57:11.026ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
65582025-06-03T00:57:11.026ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
65592025-06-03T00:57:11.026ZDEBGnexus (BackgroundTasks): client response SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "ec167925-39e4-44a4-899d-312cf4ec6984", "content-length": "372", "date": "Tue, 03 Jun 2025 00:57:11 GMT"} })
65602025-06-03T00:57:11.027ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
65612025-06-03T00:57:11.027ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
65622025-06-03T00:57:11.030ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 525.824927ms iteration = 13 status = null
65632025-06-03T00:57:11.033ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 4ec93699-353f-4b9e-ae06-25355fdb069e original_time_created = 2025-06-03 00:51:07.059326 UTC target_id = 4ec93699-353f-4b9e-ae06-25355fdb069e time_created = 2025-06-03 00:51:07.059326 UTC
65642025-06-03T00:57:11.033ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 528.882983ms iteration = 37 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"4ec93699-353f-4b9e-ae06-25355fdb069e","time_created":"2025-06-03T00:51:07.059326Z"}
65652025-06-03T00:57:11.058ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
65662025-06-03T00:57:11.060ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
65672025-06-03T00:57:11.060ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 9e79f9b1-a4de-4f9d-a901-16cc4f79e15d dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
65682025-06-03T00:57:11.060ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
65692025-06-03T00:57:11.060ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
65702025-06-03T00:57:11.061ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
65712025-06-03T00:57:11.062ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
65722025-06-03T00:57:11.062ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
65732025-06-03T00:57:11.062ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
65742025-06-03T00:57:11.062ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 558.336527ms iteration = 13 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
65752025-06-03T00:57:11.064ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: afe89ed7-19b7-479b-81eb-05c988ebc66d, lookup_type: ById(afe89ed7-19b7-479b-81eb-05c988ebc66d) } result = Ok(())
65762025-06-03T00:57:11.064ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
65772025-06-03T00:57:11.064ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
65782025-06-03T00:57:11.066ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
65792025-06-03T00:57:11.066ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
65802025-06-03T00:57:11.066ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 561.720762ms iteration = 13 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
65812025-06-03T00:57:11.070ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
65822025-06-03T00:57:11.072ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
65832025-06-03T00:57:11.073ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
65842025-06-03T00:57:11.073ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
65852025-06-03T00:57:11.074ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 569.918917ms iteration = 13 status = 0
65862025-06-03T00:57:11.084ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
65872025-06-03T00:57:11.085ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
65882025-06-03T00:57:11.088ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: ec5e33f9-c697-46e2-a93e-a745336bf9dc, rack_id: d6831ba0-55f0-4191-8dc2-ea15e55ab0ae, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(9378a284-0ee9-48e2-80a0-3bcaadb1c4e2) } rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae switch_location = Switch0
65892025-06-03T00:57:11.088ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
65902025-06-03T00:57:11.088ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
65912025-06-03T00:57:11.098ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
65922025-06-03T00:57:11.098ZDEBGnexus (BackgroundTasks): client request SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
65932025-06-03T00:57:11.099ZDEBGnexus (BackgroundTasks): client response SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "cd7ef58b-9b1c-4cb8-bd02-498613caac99", "date": "Tue, 03 Jun 2025 00:57:11 GMT"} })
65942025-06-03T00:57:11.099ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
65952025-06-03T00:57:11.102ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
65962025-06-03T00:57:11.105ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
65972025-06-03T00:57:11.105ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
65982025-06-03T00:57:11.110ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
65992025-06-03T00:57:11.110ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
66002025-06-03T00:57:11.111ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1832 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
66012025-06-03T00:57:11.111ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1859 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 9378a284-0ee9-48e2-80a0-3bcaadb1c4e2
66022025-06-03T00:57:11.111ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
66032025-06-03T00:57:11.111ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
66042025-06-03T00:57:11.111ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
66052025-06-03T00:57:11.111ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
66062025-06-03T00:57:11.111ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
66072025-06-03T00:57:11.111ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2006 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
66082025-06-03T00:57:11.111ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:408 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
66092025-06-03T00:57:11.111ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2042 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
66102025-06-03T00:57:11.111ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae
66112025-06-03T00:57:11.111ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
66122025-06-03T00:57:11.112ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
66132025-06-03T00:57:11.113ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae switch_location = Switch0
66142025-06-03T00:57:11.113ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
66152025-06-03T00:57:11.115ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
66162025-06-03T00:57:11.115ZINFOnexus (BackgroundTasks): VPC route manager sled afe89ed7-19b7-479b-81eb-05c988ebc66d push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
66172025-06-03T00:57:11.115ZDEBGnexus (BackgroundTasks): client request SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
66182025-06-03T00:57:11.116ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
66192025-06-03T00:57:11.116ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 3ef110b4-9400-4c37-8d7f-5552c8a94db0 dns_zone_name = oxide.test found_so_far = 0 version = 2
66202025-06-03T00:57:11.116ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
66212025-06-03T00:57:11.116ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
66222025-06-03T00:57:11.116ZDEBGnexus (BackgroundTasks): client response SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "54d5f2f1-f09a-4505-988b-fc8bf371e44d", "date": "Tue, 03 Jun 2025 00:57:10 GMT"} })
66232025-06-03T00:57:11.118ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 614.098239ms iteration = 13 status = {}
66242025-06-03T00:57:11.118ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
66252025-06-03T00:57:11.118ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
66262025-06-03T00:57:11.118ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
66272025-06-03T00:57:11.121ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 9e79f9b1-a4de-4f9d-a901-16cc4f79e15d dns_zone_name = control-plane.oxide.internal found_so_far = 59 version = 1
66282025-06-03T00:57:11.121ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
66292025-06-03T00:57:11.121ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2025-06-03 00:51:38.152773 UTC dns_group = internal generation = 1
66302025-06-03T00:57:11.121ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 617.873226ms iteration = 7 status = {"generation":1}
66312025-06-03T00:57:11.121ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-06-03T00:51:07.585932Z, time_modified: 2025-06-03T00:51:07.585932Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: ea0ef520-6bf3-4fbd-9ff0-f3698b48b749, db_silo: Silo { identity: SiloIdentity { id: ea0ef520-6bf3-4fbd-9ff0-f3698b48b749, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-06-03T00:51:38.523369Z, time_modified: 2025-06-03T00:51:38.523369Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "bcda8a9f16e63c3cec49f9759478b9f335a661639158b0a9026f69b846d4e127" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: ea0ef520-6bf3-4fbd-9ff0-f3698b48b749, db_silo: Silo { identity: SiloIdentity { id: ea0ef520-6bf3-4fbd-9ff0-f3698b48b749, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-06-03T00:51:38.523369Z, time_modified: 2025-06-03T00:51:38.523369Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "bcda8a9f16e63c3cec49f9759478b9f335a661639158b0a9026f69b846d4e127" }] }) }
66322025-06-03T00:57:11.121ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 617.05593ms iteration = 7 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"ea0ef520-6bf3-4fbd-9ff0-f3698b48b749","tls_certs":["bcda8a9f16e63c3cec49f9759478b9f335a661639158b0a9026f69b846d4e127"]}},"default_endpoint":{"silo_id":"ea0ef520-6bf3-4fbd-9ff0-f3698b48b749","tls_certs":["bcda8a9f16e63c3cec49f9759478b9f335a661639158b0a9026f69b846d4e127"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
66332025-06-03T00:57:11.142ZINFOnexus (BackgroundTasks): instance reincarnation completed successfully background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:113 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0
66342025-06-03T00:57:11.142ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 637.741362ms iteration = 7 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
66352025-06-03T00:57:11.148ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
66362025-06-03T00:57:11.149ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
66372025-06-03T00:57:11.150ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
66382025-06-03T00:57:11.150ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
66392025-06-03T00:57:11.152ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
66402025-06-03T00:57:11.152ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
66412025-06-03T00:57:11.153ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
66422025-06-03T00:57:11.154ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
66432025-06-03T00:57:11.154ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
66442025-06-03T00:57:11.154ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
66452025-06-03T00:57:11.154ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
66462025-06-03T00:57:11.155ZDEBGnexus (producer-registration-task): registering as oximeter producer with Nexus registration_count = 2
66472025-06-03T00:57:11.155ZDEBGnexus (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::a]:12221
66482025-06-03T00:57:11.156ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 3ef110b4-9400-4c37-8d7f-5552c8a94db0 dns_zone_name = oxide.test found_so_far = 4 version = 2
66492025-06-03T00:57:11.156ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
66502025-06-03T00:57:11.156ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2025-06-03 00:51:38.567192 UTC dns_group = external generation = 2
66512025-06-03T00:57:11.156ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 651.275393ms iteration = 7 status = {"generation":2}
66522025-06-03T00:57:11.156ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
66532025-06-03T00:57:11.156ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
66542025-06-03T00:57:11.156ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
66552025-06-03T00:57:11.187ZDEBGnexus (producer-registration-task): client request body = Some(Body) method = POST uri = http://[fd00:1122:3344:101::a]:12221/metrics/producers
66562025-06-03T00:57:11.187ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:1025 local_addr = [fd00:1122:3344:101::a]:12221 remote_addr = [fd00:1122:3344:101::a]:63627
66572025-06-03T00:57:11.187ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = POST remote_addr = [fd00:1122:3344:101::a]:63627 req_id = 14f0e1b9-95a6-431d-8fdc-c24b222eac64 roles = RoleSet { roles: {} } uri = /metrics/producers
66582025-06-03T00:57:11.188ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = POST remote_addr = [fd00:1122:3344:101::a]:63627 req_id = 14f0e1b9-95a6-431d-8fdc-c24b222eac64 resource = Database result = Ok(()) uri = /metrics/producers
66592025-06-03T00:57:11.228ZINFOnexus: assigned collector to new producer collector_id = 292a19e1-4305-4e49-b11e-08ac74714bf3 file = nexus/src/app/oximeter.rs:76 producer_id = a2861392-5fd2-462b-84e5-6f36c716263a
66602025-06-03T00:57:11.228ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 40891 local_addr = [fd00:1122:3344:101::a]:12221 method = POST remote_addr = [fd00:1122:3344:101::a]:63627 req_id = 14f0e1b9-95a6-431d-8fdc-c24b222eac64 response_code = 201 uri = /metrics/producers
66612025-06-03T00:57:11.228ZDEBGnexus (producer-registration-task): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::a]:12221/metrics/producers", status: 201, headers: {"content-type": "application/json", "x-request-id": "14f0e1b9-95a6-431d-8fdc-c24b222eac64", "content-length": "41", "date": "Tue, 03 Jun 2025 00:57:10 GMT"} })
66622025-06-03T00:57:11.229ZDEBGnexus (producer-registration-task): successfully registered with Nexus, pausing until time to renew lease lease_duration = 600s wait_period = 150s
66632025-06-03T00:57:11.250ZDEBGnexus (BackgroundTasks): client request SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
66642025-06-03T00:57:11.251ZDEBGnexus (BackgroundTasks): client response SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "0933820e-ffcd-4ad2-9e08-e0c7f3e2c64b", "content-length": "691", "date": "Tue, 03 Jun 2025 00:57:11 GMT"} })
66652025-06-03T00:57:11.251ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
66662025-06-03T00:57:11.251ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
66672025-06-03T00:57:11.251ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
66682025-06-03T00:57:11.251ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 747.440499ms iteration = 13 status = {}
66692025-06-03T00:57:11.262ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "ef7ecb66-e569-467f-befb-8667ea2975be", "date": "Tue, 03 Jun 2025 00:57:11 GMT"} })
66702025-06-03T00:57:11.262ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae switch = Switch0
66712025-06-03T00:57:11.262ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
66722025-06-03T00:57:11.263ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "d910e6ed-52f2-490d-9ee8-cf5e26deb970", "content-length": "682", "date": "Tue, 03 Jun 2025 00:57:11 GMT"} })
66732025-06-03T00:57:11.263ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
66742025-06-03T00:57:11.263ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
66752025-06-03T00:57:11.267ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
66762025-06-03T00:57:11.267ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
66772025-06-03T00:57:11.271ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
66782025-06-03T00:57:11.271ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
66792025-06-03T00:57:11.273ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
66802025-06-03T00:57:11.273ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
66812025-06-03T00:57:11.280ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 775.552611ms iteration = 13 status = {}
66822025-06-03T00:57:15.672ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 435 local_addr = [fd00:1122:3344:101::a]:63576 method = GET remote_addr = [fd00:1122:3344:101::d]:61315 req_id = ae2c5b44-8b4f-4151-9f44-e59bb8a3b2cc response_code = 200 uri = /a2861392-5fd2-462b-84e5-6f36c716263a
66832025-06-03T00:57:20.504ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 38 reason = Timeout
66842025-06-03T00:57:20.504ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 38 reason = Timeout
66852025-06-03T00:57:20.504ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
66862025-06-03T00:57:20.504ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
66872025-06-03T00:57:20.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
66882025-06-03T00:57:20.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
66892025-06-03T00:57:20.506ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
66902025-06-03T00:57:20.508ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
66912025-06-03T00:57:20.508ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
66922025-06-03T00:57:20.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
66932025-06-03T00:57:20.509ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
66942025-06-03T00:57:20.510ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
66952025-06-03T00:57:20.511ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
66962025-06-03T00:57:20.513ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 4ec93699-353f-4b9e-ae06-25355fdb069e, lookup_type: ById(4ec93699-353f-4b9e-ae06-25355fdb069e) } result = Ok(())
66972025-06-03T00:57:20.513ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
66982025-06-03T00:57:20.513ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
66992025-06-03T00:57:20.523ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 4ec93699-353f-4b9e-ae06-25355fdb069e original_time_created = 2025-06-03 00:51:07.059326 UTC target_id = 4ec93699-353f-4b9e-ae06-25355fdb069e time_created = 2025-06-03 00:51:07.059326 UTC
67002025-06-03T00:57:20.523ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 18.99864ms iteration = 38 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"4ec93699-353f-4b9e-ae06-25355fdb069e","time_created":"2025-06-03T00:51:07.059326Z"}
67012025-06-03T00:57:20.536ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
67022025-06-03T00:57:20.536ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
67032025-06-03T00:57:20.569ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 64.72681ms iteration = 38 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
67042025-06-03T00:57:25.661ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:50103 req_id = 0e8b8e50-6652-49a1-8014-cbef539a3533 roles = RoleSet { roles: {} } uri = /metrics/collectors/292a19e1-4305-4e49-b11e-08ac74714bf3/producers?sort_by=id_ascending
67052025-06-03T00:57:25.661ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:50103 req_id = 0e8b8e50-6652-49a1-8014-cbef539a3533 resource = Database result = Ok(()) uri = /metrics/collectors/292a19e1-4305-4e49-b11e-08ac74714bf3/producers?sort_by=id_ascending
67062025-06-03T00:57:25.664ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 3187 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:50103 req_id = 0e8b8e50-6652-49a1-8014-cbef539a3533 response_code = 200 uri = /metrics/collectors/292a19e1-4305-4e49-b11e-08ac74714bf3/producers?sort_by=id_ascending
67072025-06-03T00:57:25.664ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:50103 req_id = cc36c4b5-ffcc-4a45-a9e2-6ffd54bbfd1d roles = RoleSet { roles: {} } uri = /metrics/collectors/292a19e1-4305-4e49-b11e-08ac74714bf3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmZTEyZmZjMS1mMmFjLTRiZjMtOTY3Mi1mYzE4NWY4MmM1OWIifX0%3D
67082025-06-03T00:57:25.664ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:50103 req_id = cc36c4b5-ffcc-4a45-a9e2-6ffd54bbfd1d resource = Database result = Ok(()) uri = /metrics/collectors/292a19e1-4305-4e49-b11e-08ac74714bf3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmZTEyZmZjMS1mMmFjLTRiZjMtOTY3Mi1mYzE4NWY4MmM1OWIifX0%3D
67092025-06-03T00:57:25.666ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2132 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:50103 req_id = cc36c4b5-ffcc-4a45-a9e2-6ffd54bbfd1d response_code = 200 uri = /metrics/collectors/292a19e1-4305-4e49-b11e-08ac74714bf3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmZTEyZmZjMS1mMmFjLTRiZjMtOTY3Mi1mYzE4NWY4MmM1OWIifX0%3D
67102025-06-03T00:57:25.672ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 284 local_addr = [fd00:1122:3344:101::a]:63576 method = GET remote_addr = [fd00:1122:3344:101::d]:61315 req_id = 10a5d8ab-be4c-457a-8f73-851ed41a29bd response_code = 200 uri = /a2861392-5fd2-462b-84e5-6f36c716263a
67112025-06-03T00:57:30.503ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 39 reason = Timeout
67122025-06-03T00:57:30.503ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
67132025-06-03T00:57:30.503ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 39 reason = Timeout
67142025-06-03T00:57:30.503ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
67152025-06-03T00:57:30.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
67162025-06-03T00:57:30.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
67172025-06-03T00:57:30.506ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
67182025-06-03T00:57:30.509ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
67192025-06-03T00:57:30.509ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
67202025-06-03T00:57:30.509ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
67212025-06-03T00:57:30.511ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
67222025-06-03T00:57:30.511ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
67232025-06-03T00:57:30.513ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
67242025-06-03T00:57:30.515ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 4ec93699-353f-4b9e-ae06-25355fdb069e, lookup_type: ById(4ec93699-353f-4b9e-ae06-25355fdb069e) } result = Ok(())
67252025-06-03T00:57:30.515ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
67262025-06-03T00:57:30.515ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
67272025-06-03T00:57:30.527ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 4ec93699-353f-4b9e-ae06-25355fdb069e original_time_created = 2025-06-03 00:51:07.059326 UTC target_id = 4ec93699-353f-4b9e-ae06-25355fdb069e time_created = 2025-06-03 00:51:07.059326 UTC
67282025-06-03T00:57:30.527ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 23.406117ms iteration = 39 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"4ec93699-353f-4b9e-ae06-25355fdb069e","time_created":"2025-06-03T00:51:07.059326Z"}
67292025-06-03T00:57:30.540ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
67302025-06-03T00:57:30.540ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
67312025-06-03T00:57:30.573ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 69.758396ms iteration = 39 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
67322025-06-03T00:57:35.672ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 293 local_addr = [fd00:1122:3344:101::a]:63576 method = GET remote_addr = [fd00:1122:3344:101::d]:61315 req_id = 06148956-70fb-43cd-9636-0b872b96dd7a response_code = 200 uri = /a2861392-5fd2-462b-84e5-6f36c716263a
67332025-06-03T00:57:40.504ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 40 reason = Timeout
67342025-06-03T00:57:40.504ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 14 reason = Timeout
67352025-06-03T00:57:40.504ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
67362025-06-03T00:57:40.504ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 14 reason = Timeout
67372025-06-03T00:57:40.504ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
67382025-06-03T00:57:40.504ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
67392025-06-03T00:57:40.504ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 14 reason = Timeout
67402025-06-03T00:57:40.504ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
67412025-06-03T00:57:40.504ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 14 reason = Timeout
67422025-06-03T00:57:40.504ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 14 reason = Timeout
67432025-06-03T00:57:40.504ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
67442025-06-03T00:57:40.504ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
67452025-06-03T00:57:40.504ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 40 reason = Timeout
67462025-06-03T00:57:40.504ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 14 reason = Timeout
67472025-06-03T00:57:40.504ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
67482025-06-03T00:57:40.504ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
67492025-06-03T00:57:40.504ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 14 reason = Timeout
67502025-06-03T00:57:40.504ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
67512025-06-03T00:57:40.504ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
67522025-06-03T00:57:40.504ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 14 reason = Timeout
67532025-06-03T00:57:40.504ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 14 reason = Timeout
67542025-06-03T00:57:40.504ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
67552025-06-03T00:57:40.504ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
67562025-06-03T00:57:40.504ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 14 reason = Timeout
67572025-06-03T00:57:40.504ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
67582025-06-03T00:57:40.504ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 14 reason = Timeout
67592025-06-03T00:57:40.504ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 14 reason = Timeout
67602025-06-03T00:57:40.504ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
67612025-06-03T00:57:40.504ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 14 reason = Timeout
67622025-06-03T00:57:40.504ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
67632025-06-03T00:57:40.504ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 14 reason = Timeout
67642025-06-03T00:57:40.504ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
67652025-06-03T00:57:40.504ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 15 reason = Timeout
67662025-06-03T00:57:40.504ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
67672025-06-03T00:57:40.504ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 14 reason = Timeout
67682025-06-03T00:57:40.504ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
67692025-06-03T00:57:40.504ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 14 reason = Timeout
67702025-06-03T00:57:40.504ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
67712025-06-03T00:57:40.504ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
67722025-06-03T00:57:40.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
67732025-06-03T00:57:40.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
67742025-06-03T00:57:40.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
67752025-06-03T00:57:40.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
67762025-06-03T00:57:40.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
67772025-06-03T00:57:40.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
67782025-06-03T00:57:40.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
67792025-06-03T00:57:40.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
67802025-06-03T00:57:40.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
67812025-06-03T00:57:40.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
67822025-06-03T00:57:40.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
67832025-06-03T00:57:40.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
67842025-06-03T00:57:40.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
67852025-06-03T00:57:40.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
67862025-06-03T00:57:40.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
67872025-06-03T00:57:40.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
67882025-06-03T00:57:40.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
67892025-06-03T00:57:40.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
67902025-06-03T00:57:40.507ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-afe89ed7-19b7-479b-81eb-05c988ebc66d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-afe89ed7-19b7-479b-81eb-05c988ebc66d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 2787, tv_nsec: 733757014 } })
67912025-06-03T00:57:40.507ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.629523ms iteration = 14 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
67922025-06-03T00:57:40.508ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1179
67932025-06-03T00:57:40.508ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
67942025-06-03T00:57:40.509ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
67952025-06-03T00:57:40.511ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
67962025-06-03T00:57:40.511ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
67972025-06-03T00:57:40.511ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
67982025-06-03T00:57:40.511ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
67992025-06-03T00:57:40.511ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
68002025-06-03T00:57:40.511ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
68012025-06-03T00:57:40.514ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
68022025-06-03T00:57:40.517ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
68032025-06-03T00:57:40.517ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
68042025-06-03T00:57:40.517ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
68052025-06-03T00:57:40.519ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
68062025-06-03T00:57:40.519ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
68072025-06-03T00:57:40.521ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
68082025-06-03T00:57:40.521ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-afe89ed7-19b7-479b-81eb-05c988ebc66d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-afe89ed7-19b7-479b-81eb-05c988ebc66d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 2787, tv_nsec: 747443039 } })
68092025-06-03T00:57:40.521ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1179
68102025-06-03T00:57:40.522ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-afe89ed7-19b7-479b-81eb-05c988ebc66d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-afe89ed7-19b7-479b-81eb-05c988ebc66d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 2787, tv_nsec: 748622156 } })
68112025-06-03T00:57:40.522ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
68122025-06-03T00:57:40.523ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1179 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae
68132025-06-03T00:57:40.523ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
68142025-06-03T00:57:40.523ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
68152025-06-03T00:57:40.524ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
68162025-06-03T00:57:40.524ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
68172025-06-03T00:57:40.524ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
68182025-06-03T00:57:40.525ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
68192025-06-03T00:57:40.525ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
68202025-06-03T00:57:40.525ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
68212025-06-03T00:57:40.537ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
68222025-06-03T00:57:40.538ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
68232025-06-03T00:57:40.550ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
68242025-06-03T00:57:40.554ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 4ec93699-353f-4b9e-ae06-25355fdb069e, lookup_type: ById(4ec93699-353f-4b9e-ae06-25355fdb069e) } result = Ok(())
68252025-06-03T00:57:40.554ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
68262025-06-03T00:57:40.555ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
68272025-06-03T00:57:40.557ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
68282025-06-03T00:57:40.558ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
68292025-06-03T00:57:40.564ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
68302025-06-03T00:57:40.567ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
68312025-06-03T00:57:40.567ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
68322025-06-03T00:57:40.567ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
68332025-06-03T00:57:40.570ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 66.126913ms iteration = 14 status = {"errors":[],"finish_invoked_ok":[]}
68342025-06-03T00:57:40.570ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 66.700322ms iteration = 14 status = {"errors":[],"requests_created_ok":[]}
68352025-06-03T00:57:40.571ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 4ec93699-353f-4b9e-ae06-25355fdb069e original_time_created = 2025-06-03 00:51:07.059326 UTC target_id = 4ec93699-353f-4b9e-ae06-25355fdb069e time_created = 2025-06-03 00:51:07.059326 UTC
68362025-06-03T00:57:40.571ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 67.056762ms iteration = 40 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"4ec93699-353f-4b9e-ae06-25355fdb069e","time_created":"2025-06-03T00:51:07.059326Z"}
68372025-06-03T00:57:40.574ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
68382025-06-03T00:57:40.575ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
68392025-06-03T00:57:40.575ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
68402025-06-03T00:57:40.575ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
68412025-06-03T00:57:40.575ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
68422025-06-03T00:57:40.575ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
68432025-06-03T00:57:40.576ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
68442025-06-03T00:57:40.576ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
68452025-06-03T00:57:40.576ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
68462025-06-03T00:57:40.576ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
68472025-06-03T00:57:40.577ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
68482025-06-03T00:57:40.577ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
68492025-06-03T00:57:40.577ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
68502025-06-03T00:57:40.578ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
68512025-06-03T00:57:40.578ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
68522025-06-03T00:57:40.578ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
68532025-06-03T00:57:40.578ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
68542025-06-03T00:57:40.578ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
68552025-06-03T00:57:40.578ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
68562025-06-03T00:57:40.578ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
68572025-06-03T00:57:40.578ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
68582025-06-03T00:57:40.578ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
68592025-06-03T00:57:40.580ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 76.497065ms iteration = 14 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
68602025-06-03T00:57:40.581ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
68612025-06-03T00:57:40.581ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
68622025-06-03T00:57:40.583ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
68632025-06-03T00:57:40.584ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 80.5146ms iteration = 14 status = {"errors":[],"garbage_collect_requested":[]}
68642025-06-03T00:57:40.586ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
68652025-06-03T00:57:40.586ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
68662025-06-03T00:57:40.586ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
68672025-06-03T00:57:40.588ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1187 zone_address = fd00:1122:3344:101::2
68682025-06-03T00:57:40.588ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
68692025-06-03T00:57:40.588ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "06c21228-5099-4488-b57b-4e3a7f375918", "content-length": "26", "date": "Tue, 03 Jun 2025 00:57:40 GMT"} })
68702025-06-03T00:57:40.588ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1190 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
68712025-06-03T00:57:40.590ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1224 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
68722025-06-03T00:57:40.611ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 107.491797ms iteration = 40 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
68732025-06-03T00:57:40.615ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:547 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
68742025-06-03T00:57:40.615ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 111.335728ms iteration = 14 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
68752025-06-03T00:57:40.620ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
68762025-06-03T00:57:40.620ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
68772025-06-03T00:57:40.620ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
68782025-06-03T00:57:40.620ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
68792025-06-03T00:57:40.626ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 122.05998ms iteration = 14 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
68802025-06-03T00:57:40.627ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
68812025-06-03T00:57:40.627ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
68822025-06-03T00:57:40.627ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
68832025-06-03T00:57:40.628ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
68842025-06-03T00:57:40.628ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
68852025-06-03T00:57:40.629ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
68862025-06-03T00:57:40.630ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: afe89ed7-19b7-479b-81eb-05c988ebc66d, lookup_type: ById(afe89ed7-19b7-479b-81eb-05c988ebc66d) } result = Ok(())
68872025-06-03T00:57:40.630ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
68882025-06-03T00:57:40.630ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
68892025-06-03T00:57:40.631ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
68902025-06-03T00:57:40.631ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
68912025-06-03T00:57:40.631ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
68922025-06-03T00:57:40.633ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 129.344206ms iteration = 14 status = 0
68932025-06-03T00:57:40.667ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
68942025-06-03T00:57:40.667ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
68952025-06-03T00:57:40.667ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 162.93255ms iteration = 14 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
68962025-06-03T00:57:40.675ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 170.943268ms iteration = 14 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
68972025-06-03T00:57:40.680ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:372
68982025-06-03T00:57:40.680ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 176.026457ms iteration = 14 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
68992025-06-03T00:57:40.683ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
69002025-06-03T00:57:40.683ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
69012025-06-03T00:57:40.684ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
69022025-06-03T00:57:40.686ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
69032025-06-03T00:57:40.686ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
69042025-06-03T00:57:40.686ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
69052025-06-03T00:57:40.688ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 184.399236ms iteration = 14 status = {}
69062025-06-03T00:57:40.704ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 200.538187ms iteration = 15 status = {"physical_disks_added":0}
69072025-06-03T00:57:40.724ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1187 zone_address = fd00:1122:3344:101::2
69082025-06-03T00:57:40.724ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
69092025-06-03T00:57:40.724ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "e610ac2a-5f84-4b09-b602-fbef3d7f6dc4", "content-length": "26", "date": "Tue, 03 Jun 2025 00:57:40 GMT"} })
69102025-06-03T00:57:40.724ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1190 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
69112025-06-03T00:57:40.726ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1224 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
69122025-06-03T00:57:40.736ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1187 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae zone_address = fd00:1122:3344:101::2
69132025-06-03T00:57:40.736ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
69142025-06-03T00:57:40.736ZINFOnexus (BackgroundTasks): VPC route manager sled afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
69152025-06-03T00:57:40.736ZDEBGnexus (BackgroundTasks): client request SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
69162025-06-03T00:57:40.736ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "f5ad149a-fe50-44d3-8864-74048041515e", "content-length": "26", "date": "Tue, 03 Jun 2025 00:57:40 GMT"} })
69172025-06-03T00:57:40.736ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1190 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
69182025-06-03T00:57:40.737ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1224 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae
69192025-06-03T00:57:40.753ZDEBGnexus (BackgroundTasks): client response SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "707590ac-4f4d-4bfe-9fe4-298497291d2e", "content-length": "372", "date": "Tue, 03 Jun 2025 00:57:40 GMT"} })
69202025-06-03T00:57:40.753ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
69212025-06-03T00:57:40.753ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
69222025-06-03T00:57:40.776ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
69232025-06-03T00:57:40.778ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
69242025-06-03T00:57:40.778ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
69252025-06-03T00:57:40.778ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
69262025-06-03T00:57:40.791ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
69272025-06-03T00:57:40.791ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
69282025-06-03T00:57:40.793ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 289.058016ms iteration = 14 status = null
69292025-06-03T00:57:40.801ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
69302025-06-03T00:57:40.801ZDEBGnexus (BackgroundTasks): client request SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
69312025-06-03T00:57:40.801ZDEBGnexus (BackgroundTasks): client response SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "a0124443-834c-4fbc-92b1-99e47129e24c", "date": "Tue, 03 Jun 2025 00:57:40 GMT"} })
69322025-06-03T00:57:40.802ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
69332025-06-03T00:57:40.802ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
69342025-06-03T00:57:40.804ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
69352025-06-03T00:57:40.805ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
69362025-06-03T00:57:40.842ZDEBGnexus (BackgroundTasks): client request SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
69372025-06-03T00:57:40.842ZDEBGnexus (BackgroundTasks): client response SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "a905402d-8600-44c5-8e3c-a27aada1cdc6", "content-length": "691", "date": "Tue, 03 Jun 2025 00:57:40 GMT"} })
69382025-06-03T00:57:40.842ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
69392025-06-03T00:57:40.842ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
69402025-06-03T00:57:40.842ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
69412025-06-03T00:57:40.843ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 339.387669ms iteration = 14 status = {}
69422025-06-03T00:57:40.845ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
69432025-06-03T00:57:40.845ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
69442025-06-03T00:57:40.848ZINFOnexus (BackgroundTasks): VPC route manager sled afe89ed7-19b7-479b-81eb-05c988ebc66d push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
69452025-06-03T00:57:40.848ZDEBGnexus (BackgroundTasks): client request SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
69462025-06-03T00:57:40.881ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
69472025-06-03T00:57:40.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
69482025-06-03T00:57:40.884ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: ec5e33f9-c697-46e2-a93e-a745336bf9dc, rack_id: d6831ba0-55f0-4191-8dc2-ea15e55ab0ae, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(9378a284-0ee9-48e2-80a0-3bcaadb1c4e2) } rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae switch_location = Switch0
69492025-06-03T00:57:40.884ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
69502025-06-03T00:57:40.884ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
69512025-06-03T00:57:40.894ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1832 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
69522025-06-03T00:57:40.894ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1859 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 9378a284-0ee9-48e2-80a0-3bcaadb1c4e2
69532025-06-03T00:57:40.894ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
69542025-06-03T00:57:40.894ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
69552025-06-03T00:57:40.894ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
69562025-06-03T00:57:40.894ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
69572025-06-03T00:57:40.894ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
69582025-06-03T00:57:40.894ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2006 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
69592025-06-03T00:57:40.894ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:408 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
69602025-06-03T00:57:40.894ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2042 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
69612025-06-03T00:57:40.895ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae
69622025-06-03T00:57:40.895ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
69632025-06-03T00:57:40.895ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
69642025-06-03T00:57:40.897ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae switch_location = Switch0
69652025-06-03T00:57:40.897ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
69662025-06-03T00:57:40.946ZDEBGnexus (BackgroundTasks): client response SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "466bdd9c-3bbe-4b4c-89a7-8c6f357f5f0f", "date": "Tue, 03 Jun 2025 00:57:40 GMT"} })
69672025-06-03T00:57:40.947ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 443.650366ms iteration = 14 status = {}
69682025-06-03T00:57:41.039ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "f8740534-730c-47e8-91d1-f2b99497d2ab", "date": "Tue, 03 Jun 2025 00:57:40 GMT"} })
69692025-06-03T00:57:41.039ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae switch = Switch0
69702025-06-03T00:57:41.039ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
69712025-06-03T00:57:41.039ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "8b10cd4c-4bda-499b-a9b4-ba4e9882e74a", "content-length": "682", "date": "Tue, 03 Jun 2025 00:57:40 GMT"} })
69722025-06-03T00:57:41.039ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
69732025-06-03T00:57:41.039ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
69742025-06-03T00:57:41.043ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
69752025-06-03T00:57:41.044ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
69762025-06-03T00:57:41.047ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
69772025-06-03T00:57:41.047ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
69782025-06-03T00:57:41.049ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
69792025-06-03T00:57:41.049ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
69802025-06-03T00:57:41.055ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 551.638299ms iteration = 14 status = {}
69812025-06-03T00:57:45.672ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 398 local_addr = [fd00:1122:3344:101::a]:63576 method = GET remote_addr = [fd00:1122:3344:101::d]:61315 req_id = 4ebfa90e-cfb3-4bdf-bbc2-6a6e378ccdab response_code = 200 uri = /a2861392-5fd2-462b-84e5-6f36c716263a
69822025-06-03T00:57:50.504ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 41 reason = Timeout
69832025-06-03T00:57:50.504ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 41 reason = Timeout
69842025-06-03T00:57:50.504ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
69852025-06-03T00:57:50.504ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
69862025-06-03T00:57:50.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
69872025-06-03T00:57:50.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
69882025-06-03T00:57:50.506ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
69892025-06-03T00:57:50.508ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
69902025-06-03T00:57:50.508ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
69912025-06-03T00:57:50.509ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
69922025-06-03T00:57:50.511ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
69932025-06-03T00:57:50.511ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
69942025-06-03T00:57:50.512ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
69952025-06-03T00:57:50.515ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 4ec93699-353f-4b9e-ae06-25355fdb069e, lookup_type: ById(4ec93699-353f-4b9e-ae06-25355fdb069e) } result = Ok(())
69962025-06-03T00:57:50.515ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
69972025-06-03T00:57:50.515ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
69982025-06-03T00:57:50.527ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 4ec93699-353f-4b9e-ae06-25355fdb069e original_time_created = 2025-06-03 00:51:07.059326 UTC target_id = 4ec93699-353f-4b9e-ae06-25355fdb069e time_created = 2025-06-03 00:51:07.059326 UTC
69992025-06-03T00:57:50.527ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 23.021305ms iteration = 41 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"4ec93699-353f-4b9e-ae06-25355fdb069e","time_created":"2025-06-03T00:51:07.059326Z"}
70002025-06-03T00:57:50.537ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
70012025-06-03T00:57:50.537ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
70022025-06-03T00:57:50.539ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 34.77189ms iteration = 41 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
70032025-06-03T00:57:55.660ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:50103 req_id = 93233f80-97d8-4231-a265-9eef42e4bc93 roles = RoleSet { roles: {} } uri = /metrics/collectors/292a19e1-4305-4e49-b11e-08ac74714bf3/producers?sort_by=id_ascending
70042025-06-03T00:57:55.661ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:50103 req_id = 93233f80-97d8-4231-a265-9eef42e4bc93 resource = Database result = Ok(()) uri = /metrics/collectors/292a19e1-4305-4e49-b11e-08ac74714bf3/producers?sort_by=id_ascending
70052025-06-03T00:57:55.663ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2680 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:50103 req_id = 93233f80-97d8-4231-a265-9eef42e4bc93 response_code = 200 uri = /metrics/collectors/292a19e1-4305-4e49-b11e-08ac74714bf3/producers?sort_by=id_ascending
70062025-06-03T00:57:55.663ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:50103 req_id = f5622f23-dca5-4572-8437-3ebc30b2b979 roles = RoleSet { roles: {} } uri = /metrics/collectors/292a19e1-4305-4e49-b11e-08ac74714bf3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmZTEyZmZjMS1mMmFjLTRiZjMtOTY3Mi1mYzE4NWY4MmM1OWIifX0%3D
70072025-06-03T00:57:55.664ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:50103 req_id = f5622f23-dca5-4572-8437-3ebc30b2b979 resource = Database result = Ok(()) uri = /metrics/collectors/292a19e1-4305-4e49-b11e-08ac74714bf3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmZTEyZmZjMS1mMmFjLTRiZjMtOTY3Mi1mYzE4NWY4MmM1OWIifX0%3D
70082025-06-03T00:57:55.665ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 1914 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:50103 req_id = f5622f23-dca5-4572-8437-3ebc30b2b979 response_code = 200 uri = /metrics/collectors/292a19e1-4305-4e49-b11e-08ac74714bf3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmZTEyZmZjMS1mMmFjLTRiZjMtOTY3Mi1mYzE4NWY4MmM1OWIifX0%3D
70092025-06-03T00:57:55.672ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 315 local_addr = [fd00:1122:3344:101::a]:63576 method = GET remote_addr = [fd00:1122:3344:101::d]:61315 req_id = 1528bc5d-599d-4ce9-9b6d-1b2bb6725511 response_code = 200 uri = /a2861392-5fd2-462b-84e5-6f36c716263a
70102025-06-03T00:58:00.503ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 42 reason = Timeout
70112025-06-03T00:58:00.503ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 42 reason = Timeout
70122025-06-03T00:58:00.503ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
70132025-06-03T00:58:00.503ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
70142025-06-03T00:58:00.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
70152025-06-03T00:58:00.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
70162025-06-03T00:58:00.505ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
70172025-06-03T00:58:00.507ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
70182025-06-03T00:58:00.507ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
70192025-06-03T00:58:00.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
70202025-06-03T00:58:00.510ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
70212025-06-03T00:58:00.510ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
70222025-06-03T00:58:00.511ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
70232025-06-03T00:58:00.514ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 4ec93699-353f-4b9e-ae06-25355fdb069e, lookup_type: ById(4ec93699-353f-4b9e-ae06-25355fdb069e) } result = Ok(())
70242025-06-03T00:58:00.514ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
70252025-06-03T00:58:00.514ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
70262025-06-03T00:58:00.525ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 4ec93699-353f-4b9e-ae06-25355fdb069e original_time_created = 2025-06-03 00:51:07.059326 UTC target_id = 4ec93699-353f-4b9e-ae06-25355fdb069e time_created = 2025-06-03 00:51:07.059326 UTC
70272025-06-03T00:58:00.525ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 21.89431ms iteration = 42 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"4ec93699-353f-4b9e-ae06-25355fdb069e","time_created":"2025-06-03T00:51:07.059326Z"}
70282025-06-03T00:58:00.541ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
70292025-06-03T00:58:00.541ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
70302025-06-03T00:58:00.572ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 68.960181ms iteration = 42 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
70312025-06-03T00:58:05.672ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 408 local_addr = [fd00:1122:3344:101::a]:63576 method = GET remote_addr = [fd00:1122:3344:101::d]:61315 req_id = 335b00d0-de42-40b8-8ae0-c28811fa046f response_code = 200 uri = /a2861392-5fd2-462b-84e5-6f36c716263a
70322025-06-03T00:58:10.502ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 8 reason = Timeout
70332025-06-03T00:58:10.502ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
70342025-06-03T00:58:10.502ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
70352025-06-03T00:58:10.502ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
70362025-06-03T00:58:10.503ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 43 reason = Timeout
70372025-06-03T00:58:10.503ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
70382025-06-03T00:58:10.503ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
70392025-06-03T00:58:10.503ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 15 reason = Timeout
70402025-06-03T00:58:10.503ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
70412025-06-03T00:58:10.503ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 15 reason = Timeout
70422025-06-03T00:58:10.503ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 15 reason = Timeout
70432025-06-03T00:58:10.503ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
70442025-06-03T00:58:10.503ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
70452025-06-03T00:58:10.503ZDEBGnexus (BackgroundTasks): activating background_task = alert_dispatcher iteration = 8 reason = Timeout
70462025-06-03T00:58:10.503ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 8 reason = Timeout
70472025-06-03T00:58:10.503ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
70482025-06-03T00:58:10.503ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
70492025-06-03T00:58:10.503ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 8 reason = Timeout
70502025-06-03T00:58:10.503ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
70512025-06-03T00:58:10.503ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
70522025-06-03T00:58:10.503ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 8 reason = Timeout
70532025-06-03T00:58:10.503ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
70542025-06-03T00:58:10.503ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 15 reason = Timeout
70552025-06-03T00:58:10.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
70562025-06-03T00:58:10.503ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
70572025-06-03T00:58:10.503ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 15 reason = Timeout
70582025-06-03T00:58:10.503ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 15 reason = Timeout
70592025-06-03T00:58:10.503ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 15 reason = Timeout
70602025-06-03T00:58:10.503ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
70612025-06-03T00:58:10.504ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
70622025-06-03T00:58:10.504ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 15 reason = Timeout
70632025-06-03T00:58:10.504ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 15 reason = Timeout
70642025-06-03T00:58:10.504ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
70652025-06-03T00:58:10.504ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
70662025-06-03T00:58:10.504ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 16 reason = Timeout
70672025-06-03T00:58:10.504ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
70682025-06-03T00:58:10.504ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 15 reason = Timeout
70692025-06-03T00:58:10.504ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
70702025-06-03T00:58:10.504ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 15 reason = Timeout
70712025-06-03T00:58:10.504ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
70722025-06-03T00:58:10.504ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 10 reason = Timeout
70732025-06-03T00:58:10.504ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 10 reason = Timeout
70742025-06-03T00:58:10.504ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 8 reason = Timeout
70752025-06-03T00:58:10.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
70762025-06-03T00:58:10.504ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-06-03 00:48:10.504090888 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
70772025-06-03T00:58:10.504ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 8 reason = Timeout
70782025-06-03T00:58:10.504ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
70792025-06-03T00:58:10.504ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 9 reason = Timeout
70802025-06-03T00:58:10.504ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:85 reason = blueprint disabled target_id = 4ec93699-353f-4b9e-ae06-25355fdb069e
70812025-06-03T00:58:10.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
70822025-06-03T00:58:10.504ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 8 reason = Timeout
70832025-06-03T00:58:10.504ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 10.69\u{b5}s iteration = 9 status = {"enabled":false,"target_id":"4ec93699-353f-4b9e-ae06-25355fdb069e"}
70842025-06-03T00:58:10.504ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 15 reason = Timeout
70852025-06-03T00:58:10.504ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
70862025-06-03T00:58:10.504ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 15 reason = Timeout
70872025-06-03T00:58:10.504ZDEBGnexus (BackgroundTasks): activating background_task = webhook_deliverator iteration = 8 reason = Timeout
70882025-06-03T00:58:10.504ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
70892025-06-03T00:58:10.504ZDEBGnexus (BackgroundTasks): roles background_task = webhook_deliverator roles = RoleSet { roles: {} }
70902025-06-03T00:58:10.504ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 8 reason = Timeout
70912025-06-03T00:58:10.504ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 8 reason = Timeout
70922025-06-03T00:58:10.504ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
70932025-06-03T00:58:10.504ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
70942025-06-03T00:58:10.504ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 15 reason = Timeout
70952025-06-03T00:58:10.504ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
70962025-06-03T00:58:10.504ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
70972025-06-03T00:58:10.504ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 15 reason = Timeout
70982025-06-03T00:58:10.504ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 8 reason = Timeout
70992025-06-03T00:58:10.504ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
71002025-06-03T00:58:10.504ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:53
71012025-06-03T00:58:10.504ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
71022025-06-03T00:58:10.504ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 15 reason = Timeout
71032025-06-03T00:58:10.504ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
71042025-06-03T00:58:10.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
71052025-06-03T00:58:10.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
71062025-06-03T00:58:10.504ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
71072025-06-03T00:58:10.504ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 43 reason = Timeout
71082025-06-03T00:58:10.504ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
71092025-06-03T00:58:10.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
71102025-06-03T00:58:10.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
71112025-06-03T00:58:10.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
71122025-06-03T00:58:10.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
71132025-06-03T00:58:10.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
71142025-06-03T00:58:10.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = alert_dispatcher resource = Database result = Ok(())
71152025-06-03T00:58:10.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
71162025-06-03T00:58:10.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
71172025-06-03T00:58:10.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
71182025-06-03T00:58:10.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
71192025-06-03T00:58:10.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
71202025-06-03T00:58:10.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
71212025-06-03T00:58:10.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
71222025-06-03T00:58:10.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
71232025-06-03T00:58:10.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
71242025-06-03T00:58:10.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
71252025-06-03T00:58:10.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
71262025-06-03T00:58:10.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
71272025-06-03T00:58:10.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = webhook_deliverator resource = Database result = Ok(())
71282025-06-03T00:58:10.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
71292025-06-03T00:58:10.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
71302025-06-03T00:58:10.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
71312025-06-03T00:58:10.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
71322025-06-03T00:58:10.504ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("9feb16c9-ff60-43e8-917b-965124f46e0d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("aa124775-31de-4648-981a-4af500268b66.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("aa124775-31de-4648-981a-4af500268b66.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::9))) }], valid_until: Instant { tv_sec: 2817, tv_nsec: 731276187 } })
71332025-06-03T00:58:10.504ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("241573bc-467c-4802-9f4b-f7c82da4e12e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("64fc0a69-d4b8-4f6c-9a07-77a4cfa94962.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("ceb74144-4b6b-4b3f-bc80-eac2d75f3bc9.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("ceb74144-4b6b-4b3f-bc80-eac2d75f3bc9.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:2::1))) }], valid_until: Instant { tv_sec: 2817, tv_nsec: 731344629 } })
71342025-06-03T00:58:10.505ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
71352025-06-03T00:58:10.505ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.318571ms iteration = 8 status = {"addresses":["[fd00:1122:3344:101::8]:5353","[fd00:1122:3344:101::9]:5353"]}
71362025-06-03T00:58:10.505ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } dns_group = internal
71372025-06-03T00:58:10.505ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.527648ms iteration = 8 status = {"addresses":["[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:3::1]:5353","[fd00:1122:3344:2::1]:5353"]}
71382025-06-03T00:58:10.510ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
71392025-06-03T00:58:10.510ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
71402025-06-03T00:58:10.510ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
71412025-06-03T00:58:10.513ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
71422025-06-03T00:58:10.513ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
71432025-06-03T00:58:10.514ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 10.128194ms iteration = 15 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
71442025-06-03T00:58:10.514ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
71452025-06-03T00:58:10.514ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
71462025-06-03T00:58:10.515ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
71472025-06-03T00:58:10.515ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
71482025-06-03T00:58:10.515ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
71492025-06-03T00:58:10.518ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
71502025-06-03T00:58:10.519ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
71512025-06-03T00:58:10.524ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
71522025-06-03T00:58:10.526ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-afe89ed7-19b7-479b-81eb-05c988ebc66d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-afe89ed7-19b7-479b-81eb-05c988ebc66d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 2817, tv_nsec: 752647491 } })
71532025-06-03T00:58:10.526ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1179
71542025-06-03T00:58:10.527ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
71552025-06-03T00:58:10.527ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
71562025-06-03T00:58:10.527ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-afe89ed7-19b7-479b-81eb-05c988ebc66d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-afe89ed7-19b7-479b-81eb-05c988ebc66d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 2817, tv_nsec: 754195799 } })
71572025-06-03T00:58:10.527ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
71582025-06-03T00:58:10.528ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1179
71592025-06-03T00:58:10.534ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-06-03 00:48:10.504090888 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
71602025-06-03T00:58:10.534ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 30.240259ms iteration = 8 status = {"expiration":"2025-06-03T00:48:10.504090888Z","pruned":[]}
71612025-06-03T00:58:10.535ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
71622025-06-03T00:58:10.537ZINFOnexus (BackgroundTasks): all webhook delivery tasks completed background_task = webhook_deliverator file = nexus/src/app/background/tasks/webhook_deliverator.rs:202 num_receivers = 0
71632025-06-03T00:58:10.537ZDEBGnexus (BackgroundTasks): activation complete background_task = webhook_deliverator elapsed = 33.726867ms iteration = 8 status = {"by_rx":{},"error":null}
71642025-06-03T00:58:10.538ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 34.019606ms iteration = 8 status = {"errors":[],"found_port_ok":[]}
71652025-06-03T00:58:10.538ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
71662025-06-03T00:58:10.538ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
71672025-06-03T00:58:10.539ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
71682025-06-03T00:58:10.542ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
71692025-06-03T00:58:10.547ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
71702025-06-03T00:58:10.552ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
71712025-06-03T00:58:10.552ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
71722025-06-03T00:58:10.567ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
71732025-06-03T00:58:10.568ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = alert_dispatcher resource = Database result = Ok(())
71742025-06-03T00:58:10.575ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
71752025-06-03T00:58:10.576ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
71762025-06-03T00:58:10.580ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-afe89ed7-19b7-479b-81eb-05c988ebc66d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-afe89ed7-19b7-479b-81eb-05c988ebc66d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 2817, tv_nsec: 806453981 } })
71772025-06-03T00:58:10.580ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1179 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae
71782025-06-03T00:58:10.581ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
71792025-06-03T00:58:10.582ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 78.774846ms iteration = 15 status = {"errors":[],"finish_invoked_ok":[]}
71802025-06-03T00:58:10.584ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
71812025-06-03T00:58:10.584ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
71822025-06-03T00:58:10.584ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
71832025-06-03T00:58:10.584ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
71842025-06-03T00:58:10.584ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
71852025-06-03T00:58:10.584ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
71862025-06-03T00:58:10.585ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
71872025-06-03T00:58:10.585ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
71882025-06-03T00:58:10.592ZDEBGnexus (BackgroundTasks): activation complete background_task = alert_dispatcher elapsed = 88.25131ms iteration = 8 status = {"dispatched":[],"errors":[],"glob_version":"144.0.0","globs_reprocessed":{},"no_receivers":[]}
71892025-06-03T00:58:10.592ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
71902025-06-03T00:58:10.594ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
71912025-06-03T00:58:10.595ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
71922025-06-03T00:58:10.595ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
71932025-06-03T00:58:10.596ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
71942025-06-03T00:58:10.598ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
71952025-06-03T00:58:10.598ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
71962025-06-03T00:58:10.598ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
71972025-06-03T00:58:10.599ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
71982025-06-03T00:58:10.599ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
71992025-06-03T00:58:10.599ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
72002025-06-03T00:58:10.600ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
72012025-06-03T00:58:10.600ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
72022025-06-03T00:58:10.600ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
72032025-06-03T00:58:10.601ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 97.739826ms iteration = 15 status = {"errors":[],"garbage_collect_requested":[]}
72042025-06-03T00:58:10.601ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
72052025-06-03T00:58:10.601ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:167 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
72062025-06-03T00:58:10.601ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 97.504187ms iteration = 8 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
72072025-06-03T00:58:10.602ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
72082025-06-03T00:58:10.602ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
72092025-06-03T00:58:10.602ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
72102025-06-03T00:58:10.602ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
72112025-06-03T00:58:10.603ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 99.975224ms iteration = 15 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
72122025-06-03T00:58:10.605ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
72132025-06-03T00:58:10.606ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 102.73371ms iteration = 15 status = {"errors":[],"requests_created_ok":[]}
72142025-06-03T00:58:10.609ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
72152025-06-03T00:58:10.609ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
72162025-06-03T00:58:10.609ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
72172025-06-03T00:58:10.612ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
72182025-06-03T00:58:10.612ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
72192025-06-03T00:58:10.619ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 115.971181ms iteration = 43 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
72202025-06-03T00:58:10.620ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
72212025-06-03T00:58:10.621ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
72222025-06-03T00:58:10.623ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
72232025-06-03T00:58:10.623ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
72242025-06-03T00:58:10.623ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
72252025-06-03T00:58:10.624ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
72262025-06-03T00:58:10.625ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
72272025-06-03T00:58:10.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
72282025-06-03T00:58:10.626ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
72292025-06-03T00:58:10.626ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
72302025-06-03T00:58:10.626ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
72312025-06-03T00:58:10.626ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
72322025-06-03T00:58:10.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
72332025-06-03T00:58:10.628ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
72342025-06-03T00:58:10.628ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
72352025-06-03T00:58:10.628ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
72362025-06-03T00:58:10.630ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
72372025-06-03T00:58:10.631ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
72382025-06-03T00:58:10.631ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
72392025-06-03T00:58:10.631ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
72402025-06-03T00:58:10.632ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
72412025-06-03T00:58:10.633ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
72422025-06-03T00:58:10.636ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
72432025-06-03T00:58:10.636ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
72442025-06-03T00:58:10.636ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
72452025-06-03T00:58:10.639ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
72462025-06-03T00:58:10.644ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 4ec93699-353f-4b9e-ae06-25355fdb069e, lookup_type: ById(4ec93699-353f-4b9e-ae06-25355fdb069e) } result = Ok(())
72472025-06-03T00:58:10.644ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
72482025-06-03T00:58:10.644ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
72492025-06-03T00:58:10.648ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
72502025-06-03T00:58:10.651ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
72512025-06-03T00:58:10.651ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
72522025-06-03T00:58:10.651ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
72532025-06-03T00:58:10.653ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
72542025-06-03T00:58:10.656ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
72552025-06-03T00:58:10.656ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
72562025-06-03T00:58:10.656ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
72572025-06-03T00:58:10.661ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
72582025-06-03T00:58:10.662ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 4ec93699-353f-4b9e-ae06-25355fdb069e original_time_created = 2025-06-03 00:51:07.059326 UTC target_id = 4ec93699-353f-4b9e-ae06-25355fdb069e time_created = 2025-06-03 00:51:07.059326 UTC
72592025-06-03T00:58:10.662ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 157.702617ms iteration = 43 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"4ec93699-353f-4b9e-ae06-25355fdb069e","time_created":"2025-06-03T00:51:07.059326Z"}
72602025-06-03T00:58:10.663ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
72612025-06-03T00:58:10.663ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
72622025-06-03T00:58:10.665ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
72632025-06-03T00:58:10.665ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
72642025-06-03T00:58:10.665ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
72652025-06-03T00:58:10.667ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:547 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
72662025-06-03T00:58:10.667ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 163.379913ms iteration = 15 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
72672025-06-03T00:58:10.668ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
72682025-06-03T00:58:10.668ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
72692025-06-03T00:58:10.668ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
72702025-06-03T00:58:10.668ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
72712025-06-03T00:58:10.668ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
72722025-06-03T00:58:10.668ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
72732025-06-03T00:58:10.668ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
72742025-06-03T00:58:10.669ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
72752025-06-03T00:58:10.670ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
72762025-06-03T00:58:10.670ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 9e79f9b1-a4de-4f9d-a901-16cc4f79e15d dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
72772025-06-03T00:58:10.670ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
72782025-06-03T00:58:10.671ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
72792025-06-03T00:58:10.671ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
72802025-06-03T00:58:10.671ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
72812025-06-03T00:58:10.673ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
72822025-06-03T00:58:10.673ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
72832025-06-03T00:58:10.673ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
72842025-06-03T00:58:10.673ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1187 zone_address = fd00:1122:3344:101::2
72852025-06-03T00:58:10.673ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
72862025-06-03T00:58:10.674ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "51478428-7290-4f47-9472-9feb9f7aa319", "content-length": "26", "date": "Tue, 03 Jun 2025 00:58:10 GMT"} })
72872025-06-03T00:58:10.674ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1190 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
72882025-06-03T00:58:10.675ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1224 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
72892025-06-03T00:58:10.676ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
72902025-06-03T00:58:10.676ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
72912025-06-03T00:58:10.677ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
72922025-06-03T00:58:10.677ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: afe89ed7-19b7-479b-81eb-05c988ebc66d, lookup_type: ById(afe89ed7-19b7-479b-81eb-05c988ebc66d) } result = Ok(())
72932025-06-03T00:58:10.677ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
72942025-06-03T00:58:10.677ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
72952025-06-03T00:58:10.678ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
72962025-06-03T00:58:10.678ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
72972025-06-03T00:58:10.678ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
72982025-06-03T00:58:10.680ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 176.282944ms iteration = 15 status = 0
72992025-06-03T00:58:10.682ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 9e79f9b1-a4de-4f9d-a901-16cc4f79e15d dns_zone_name = control-plane.oxide.internal found_so_far = 59 version = 1
73002025-06-03T00:58:10.682ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
73012025-06-03T00:58:10.682ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2025-06-03 00:51:38.152773 UTC dns_group = internal generation = 1
73022025-06-03T00:58:10.682ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 179.852695ms iteration = 8 status = {"generation":1}
73032025-06-03T00:58:10.683ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1187 zone_address = fd00:1122:3344:101::2
73042025-06-03T00:58:10.683ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
73052025-06-03T00:58:10.684ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "f2acca02-830b-496e-a77e-4b676be29354", "content-length": "26", "date": "Tue, 03 Jun 2025 00:58:10 GMT"} })
73062025-06-03T00:58:10.684ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1190 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
73072025-06-03T00:58:10.689ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1224 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
73082025-06-03T00:58:10.697ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 193.182608ms iteration = 15 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
73092025-06-03T00:58:10.701ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 197.923986ms iteration = 15 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
73102025-06-03T00:58:10.702ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
73112025-06-03T00:58:10.702ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 3ef110b4-9400-4c37-8d7f-5552c8a94db0 dns_zone_name = oxide.test found_so_far = 0 version = 2
73122025-06-03T00:58:10.702ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
73132025-06-03T00:58:10.702ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
73142025-06-03T00:58:10.704ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
73152025-06-03T00:58:10.708ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1187 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae zone_address = fd00:1122:3344:101::2
73162025-06-03T00:58:10.708ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
73172025-06-03T00:58:10.708ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "a5fed1fd-9eba-48ba-9dc2-2a5fd5f0ff5a", "content-length": "26", "date": "Tue, 03 Jun 2025 00:58:10 GMT"} })
73182025-06-03T00:58:10.708ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1190 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
73192025-06-03T00:58:10.708ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
73202025-06-03T00:58:10.708ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
73212025-06-03T00:58:10.709ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
73222025-06-03T00:58:10.710ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
73232025-06-03T00:58:10.710ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:137 found = 0 not_ready_to_be_deleted = 0
73242025-06-03T00:58:10.710ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 205.997266ms iteration = 8 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
73252025-06-03T00:58:10.713ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1224 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae
73262025-06-03T00:58:10.713ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 3ef110b4-9400-4c37-8d7f-5552c8a94db0 dns_zone_name = oxide.test found_so_far = 4 version = 2
73272025-06-03T00:58:10.713ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
73282025-06-03T00:58:10.713ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2025-06-03 00:51:38.567192 UTC dns_group = external generation = 2
73292025-06-03T00:58:10.713ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 209.678751ms iteration = 8 status = {"generation":2}
73302025-06-03T00:58:10.725ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
73312025-06-03T00:58:10.726ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
73322025-06-03T00:58:10.726ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
73332025-06-03T00:58:10.729ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
73342025-06-03T00:58:10.729ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
73352025-06-03T00:58:10.729ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
73362025-06-03T00:58:10.729ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
73372025-06-03T00:58:10.729ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
73382025-06-03T00:58:10.729ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 225.314097ms iteration = 15 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
73392025-06-03T00:58:10.730ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 226.558465ms iteration = 16 status = {"physical_disks_added":0}
73402025-06-03T00:58:10.740ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-06-03T00:51:07.585932Z, time_modified: 2025-06-03T00:51:07.585932Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: ea0ef520-6bf3-4fbd-9ff0-f3698b48b749, db_silo: Silo { identity: SiloIdentity { id: ea0ef520-6bf3-4fbd-9ff0-f3698b48b749, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-06-03T00:51:38.523369Z, time_modified: 2025-06-03T00:51:38.523369Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "bcda8a9f16e63c3cec49f9759478b9f335a661639158b0a9026f69b846d4e127" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: ea0ef520-6bf3-4fbd-9ff0-f3698b48b749, db_silo: Silo { identity: SiloIdentity { id: ea0ef520-6bf3-4fbd-9ff0-f3698b48b749, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-06-03T00:51:38.523369Z, time_modified: 2025-06-03T00:51:38.523369Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "bcda8a9f16e63c3cec49f9759478b9f335a661639158b0a9026f69b846d4e127" }] }) }
73412025-06-03T00:58:10.740ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 236.800693ms iteration = 8 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"ea0ef520-6bf3-4fbd-9ff0-f3698b48b749","tls_certs":["bcda8a9f16e63c3cec49f9759478b9f335a661639158b0a9026f69b846d4e127"]}},"default_endpoint":{"silo_id":"ea0ef520-6bf3-4fbd-9ff0-f3698b48b749","tls_certs":["bcda8a9f16e63c3cec49f9759478b9f335a661639158b0a9026f69b846d4e127"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
73422025-06-03T00:58:10.750ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
73432025-06-03T00:58:10.750ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
73442025-06-03T00:58:10.768ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
73452025-06-03T00:58:10.770ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
73462025-06-03T00:58:10.770ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "62d35f1e-fd7d-41ed-b457-a1c1b45da9b6", "date": "Tue, 03 Jun 2025 00:58:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
73472025-06-03T00:58:10.770ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
73482025-06-03T00:58:10.772ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "e5b78a4c-19aa-4442-8aea-fb9e44fd76f0", "date": "Tue, 03 Jun 2025 00:58:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
73492025-06-03T00:58:10.772ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
73502025-06-03T00:58:10.773ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
73512025-06-03T00:58:10.773ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 269.816679ms iteration = 10 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
73522025-06-03T00:58:10.777ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:372
73532025-06-03T00:58:10.777ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 273.454361ms iteration = 15 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
73542025-06-03T00:58:10.817ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 313.135213ms iteration = 15 status = {}
73552025-06-03T00:58:10.836ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
73562025-06-03T00:58:10.836ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
73572025-06-03T00:58:10.838ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 334.848248ms iteration = 15 status = null
73582025-06-03T00:58:10.880ZINFOnexus (BackgroundTasks): VPC route manager sled afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
73592025-06-03T00:58:10.881ZDEBGnexus (BackgroundTasks): client request SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
73602025-06-03T00:58:10.881ZDEBGnexus (BackgroundTasks): client response SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "b6bbc53e-9dd7-487b-8be3-fd6833345b9c", "content-length": "372", "date": "Tue, 03 Jun 2025 00:58:10 GMT"} })
73612025-06-03T00:58:10.881ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
73622025-06-03T00:58:10.881ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
73632025-06-03T00:58:10.882ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
73642025-06-03T00:58:10.882ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "96dcb1b5-8705-4f72-b005-b84f32c95c73", "date": "Tue, 03 Jun 2025 00:58:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
73652025-06-03T00:58:10.882ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
73662025-06-03T00:58:10.885ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "867c0ef7-3e98-40e2-a9b7-b3ee415619fa", "date": "Tue, 03 Jun 2025 00:58:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
73672025-06-03T00:58:10.885ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
73682025-06-03T00:58:10.885ZINFOnexus (BackgroundTasks): instance reincarnation completed successfully background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:113 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0
73692025-06-03T00:58:10.885ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 382.001822ms iteration = 8 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
73702025-06-03T00:58:10.886ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "a383cc48-3888-4288-931f-4fb025f52131", "date": "Tue, 03 Jun 2025 00:58:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
73712025-06-03T00:58:10.887ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
73722025-06-03T00:58:10.889ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
73732025-06-03T00:58:10.889ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 385.017775ms iteration = 10 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
73742025-06-03T00:58:10.895ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
73752025-06-03T00:58:10.895ZDEBGnexus (BackgroundTasks): client request SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
73762025-06-03T00:58:10.898ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
73772025-06-03T00:58:10.898ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
73782025-06-03T00:58:10.900ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
73792025-06-03T00:58:10.901ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
73802025-06-03T00:58:10.901ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
73812025-06-03T00:58:10.902ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
73822025-06-03T00:58:10.929ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
73832025-06-03T00:58:10.929ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
73842025-06-03T00:58:10.931ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: ec5e33f9-c697-46e2-a93e-a745336bf9dc, rack_id: d6831ba0-55f0-4191-8dc2-ea15e55ab0ae, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(9378a284-0ee9-48e2-80a0-3bcaadb1c4e2) } rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae switch_location = Switch0
73852025-06-03T00:58:10.931ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
73862025-06-03T00:58:10.931ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
73872025-06-03T00:58:10.934ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
73882025-06-03T00:58:10.936ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
73892025-06-03T00:58:10.936ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
73902025-06-03T00:58:10.936ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
73912025-06-03T00:58:10.945ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1832 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
73922025-06-03T00:58:10.946ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1859 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 9378a284-0ee9-48e2-80a0-3bcaadb1c4e2
73932025-06-03T00:58:10.946ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
73942025-06-03T00:58:10.946ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
73952025-06-03T00:58:10.946ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
73962025-06-03T00:58:10.946ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
73972025-06-03T00:58:10.946ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
73982025-06-03T00:58:10.946ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2006 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
73992025-06-03T00:58:10.946ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:408 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
74002025-06-03T00:58:10.946ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2042 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
74012025-06-03T00:58:10.946ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae
74022025-06-03T00:58:10.946ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
74032025-06-03T00:58:10.947ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
74042025-06-03T00:58:10.949ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae switch_location = Switch0
74052025-06-03T00:58:10.949ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
74062025-06-03T00:58:10.992ZDEBGnexus (BackgroundTasks): client response SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "26873dd7-e324-4e10-8bed-ff250e76ecf1", "date": "Tue, 03 Jun 2025 00:58:10 GMT"} })
74072025-06-03T00:58:10.992ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
74082025-06-03T00:58:10.992ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
74092025-06-03T00:58:10.994ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
74102025-06-03T00:58:10.995ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
74112025-06-03T00:58:10.998ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
74122025-06-03T00:58:10.998ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
74132025-06-03T00:58:11.001ZINFOnexus (BackgroundTasks): VPC route manager sled afe89ed7-19b7-479b-81eb-05c988ebc66d push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
74142025-06-03T00:58:11.001ZDEBGnexus (BackgroundTasks): client request SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
74152025-06-03T00:58:11.002ZDEBGnexus (BackgroundTasks): client response SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "bcb6a640-5e56-4a27-813c-54f7b202ece6", "date": "Tue, 03 Jun 2025 00:58:11 GMT"} })
74162025-06-03T00:58:11.003ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 499.105928ms iteration = 15 status = {}
74172025-06-03T00:58:11.030ZDEBGnexus (BackgroundTasks): client request SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
74182025-06-03T00:58:11.030ZDEBGnexus (BackgroundTasks): client response SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "96f6c3d5-daa9-4c08-b314-c85b0c59d699", "content-length": "691", "date": "Tue, 03 Jun 2025 00:58:11 GMT"} })
74192025-06-03T00:58:11.030ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
74202025-06-03T00:58:11.030ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
74212025-06-03T00:58:11.030ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
74222025-06-03T00:58:11.032ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 528.867142ms iteration = 15 status = {}
74232025-06-03T00:58:11.186ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "81426468-bdcc-453e-92b7-20733c2c927e", "date": "Tue, 03 Jun 2025 00:58:11 GMT"} })
74242025-06-03T00:58:11.186ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae switch = Switch0
74252025-06-03T00:58:11.187ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
74262025-06-03T00:58:11.187ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "d0f95edd-b667-456c-9f48-9747d76871a3", "content-length": "682", "date": "Tue, 03 Jun 2025 00:58:10 GMT"} })
74272025-06-03T00:58:11.187ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
74282025-06-03T00:58:11.187ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
74292025-06-03T00:58:11.191ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
74302025-06-03T00:58:11.192ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
74312025-06-03T00:58:11.196ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
74322025-06-03T00:58:11.196ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
74332025-06-03T00:58:11.198ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
74342025-06-03T00:58:11.199ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
74352025-06-03T00:58:11.205ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 701.388949ms iteration = 15 status = {}
74362025-06-03T00:58:15.671ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 412 local_addr = [fd00:1122:3344:101::a]:63576 method = GET remote_addr = [fd00:1122:3344:101::d]:61315 req_id = 0e241669-a8a8-4056-8f3b-d77cb7ab5426 response_code = 200 uri = /a2861392-5fd2-462b-84e5-6f36c716263a
74372025-06-03T00:58:20.504ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 44 reason = Timeout
74382025-06-03T00:58:20.504ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 44 reason = Timeout
74392025-06-03T00:58:20.504ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
74402025-06-03T00:58:20.504ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
74412025-06-03T00:58:20.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
74422025-06-03T00:58:20.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
74432025-06-03T00:58:20.506ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
74442025-06-03T00:58:20.508ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
74452025-06-03T00:58:20.508ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
74462025-06-03T00:58:20.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
74472025-06-03T00:58:20.511ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
74482025-06-03T00:58:20.511ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
74492025-06-03T00:58:20.512ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
74502025-06-03T00:58:20.515ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 4ec93699-353f-4b9e-ae06-25355fdb069e, lookup_type: ById(4ec93699-353f-4b9e-ae06-25355fdb069e) } result = Ok(())
74512025-06-03T00:58:20.515ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
74522025-06-03T00:58:20.515ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
74532025-06-03T00:58:20.527ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 4ec93699-353f-4b9e-ae06-25355fdb069e original_time_created = 2025-06-03 00:51:07.059326 UTC target_id = 4ec93699-353f-4b9e-ae06-25355fdb069e time_created = 2025-06-03 00:51:07.059326 UTC
74542025-06-03T00:58:20.527ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 23.101607ms iteration = 44 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"4ec93699-353f-4b9e-ae06-25355fdb069e","time_created":"2025-06-03T00:51:07.059326Z"}
74552025-06-03T00:58:20.538ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
74562025-06-03T00:58:20.538ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
74572025-06-03T00:58:20.539ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 35.821022ms iteration = 44 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
74582025-06-03T00:58:25.671ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 274 local_addr = [fd00:1122:3344:101::a]:63576 method = GET remote_addr = [fd00:1122:3344:101::d]:61315 req_id = 09ea4273-9bd3-41f8-8b4f-84bf6fdeaf41 response_code = 200 uri = /a2861392-5fd2-462b-84e5-6f36c716263a
74592025-06-03T00:58:30.503ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 45 reason = Timeout
74602025-06-03T00:58:30.503ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
74612025-06-03T00:58:30.503ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 45 reason = Timeout
74622025-06-03T00:58:30.503ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
74632025-06-03T00:58:30.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
74642025-06-03T00:58:30.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
74652025-06-03T00:58:30.505ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
74662025-06-03T00:58:30.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
74672025-06-03T00:58:30.505ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
74682025-06-03T00:58:30.507ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.297152ms iteration = 45 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
74692025-06-03T00:58:30.508ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
74702025-06-03T00:58:30.508ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
74712025-06-03T00:58:30.509ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
74722025-06-03T00:58:30.510ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
74732025-06-03T00:58:30.511ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
74742025-06-03T00:58:30.512ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
74752025-06-03T00:58:30.514ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 4ec93699-353f-4b9e-ae06-25355fdb069e, lookup_type: ById(4ec93699-353f-4b9e-ae06-25355fdb069e) } result = Ok(())
74762025-06-03T00:58:30.514ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
74772025-06-03T00:58:30.515ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
74782025-06-03T00:58:30.525ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 4ec93699-353f-4b9e-ae06-25355fdb069e original_time_created = 2025-06-03 00:51:07.059326 UTC target_id = 4ec93699-353f-4b9e-ae06-25355fdb069e time_created = 2025-06-03 00:51:07.059326 UTC
74792025-06-03T00:58:30.525ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 21.288141ms iteration = 45 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"4ec93699-353f-4b9e-ae06-25355fdb069e","time_created":"2025-06-03T00:51:07.059326Z"}
74802025-06-03T00:58:35.671ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 390 local_addr = [fd00:1122:3344:101::a]:63576 method = GET remote_addr = [fd00:1122:3344:101::d]:61315 req_id = 4424c807-673d-4401-9e6f-488325105343 response_code = 200 uri = /a2861392-5fd2-462b-84e5-6f36c716263a
74812025-06-03T00:58:40.503ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 46 reason = Timeout
74822025-06-03T00:58:40.503ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 16 reason = Timeout
74832025-06-03T00:58:40.503ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
74842025-06-03T00:58:40.503ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
74852025-06-03T00:58:40.503ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 16 reason = Timeout
74862025-06-03T00:58:40.503ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
74872025-06-03T00:58:40.503ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 16 reason = Timeout
74882025-06-03T00:58:40.503ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 16 reason = Timeout
74892025-06-03T00:58:40.503ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
74902025-06-03T00:58:40.503ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
74912025-06-03T00:58:40.503ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 16 reason = Timeout
74922025-06-03T00:58:40.503ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
74932025-06-03T00:58:40.503ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 16 reason = Timeout
74942025-06-03T00:58:40.503ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
74952025-06-03T00:58:40.503ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 16 reason = Timeout
74962025-06-03T00:58:40.503ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
74972025-06-03T00:58:40.503ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
74982025-06-03T00:58:40.503ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 16 reason = Timeout
74992025-06-03T00:58:40.503ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 16 reason = Timeout
75002025-06-03T00:58:40.503ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
75012025-06-03T00:58:40.503ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
75022025-06-03T00:58:40.503ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 16 reason = Timeout
75032025-06-03T00:58:40.503ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
75042025-06-03T00:58:40.503ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 16 reason = Timeout
75052025-06-03T00:58:40.503ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
75062025-06-03T00:58:40.503ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 46 reason = Timeout
75072025-06-03T00:58:40.503ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
75082025-06-03T00:58:40.503ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 17 reason = Timeout
75092025-06-03T00:58:40.503ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 16 reason = Timeout
75102025-06-03T00:58:40.503ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
75112025-06-03T00:58:40.503ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
75122025-06-03T00:58:40.503ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 16 reason = Timeout
75132025-06-03T00:58:40.503ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
75142025-06-03T00:58:40.503ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 16 reason = Timeout
75152025-06-03T00:58:40.503ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
75162025-06-03T00:58:40.503ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
75172025-06-03T00:58:40.503ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 16 reason = Timeout
75182025-06-03T00:58:40.503ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
75192025-06-03T00:58:40.503ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 16 reason = Timeout
75202025-06-03T00:58:40.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
75212025-06-03T00:58:40.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
75222025-06-03T00:58:40.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
75232025-06-03T00:58:40.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
75242025-06-03T00:58:40.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
75252025-06-03T00:58:40.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
75262025-06-03T00:58:40.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
75272025-06-03T00:58:40.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
75282025-06-03T00:58:40.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
75292025-06-03T00:58:40.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
75302025-06-03T00:58:40.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
75312025-06-03T00:58:40.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
75322025-06-03T00:58:40.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
75332025-06-03T00:58:40.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
75342025-06-03T00:58:40.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
75352025-06-03T00:58:40.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
75362025-06-03T00:58:40.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
75372025-06-03T00:58:40.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
75382025-06-03T00:58:40.506ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
75392025-06-03T00:58:40.507ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 3.982024ms iteration = 16 status = {"errors":[],"garbage_collect_requested":[]}
75402025-06-03T00:58:40.507ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-afe89ed7-19b7-479b-81eb-05c988ebc66d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-afe89ed7-19b7-479b-81eb-05c988ebc66d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 2847, tv_nsec: 734666252 } })
75412025-06-03T00:58:40.508ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1179
75422025-06-03T00:58:40.508ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 4.347226ms iteration = 16 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
75432025-06-03T00:58:40.510ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
75442025-06-03T00:58:40.510ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
75452025-06-03T00:58:40.510ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
75462025-06-03T00:58:40.513ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
75472025-06-03T00:58:40.513ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
75482025-06-03T00:58:40.514ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
75492025-06-03T00:58:40.514ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
75502025-06-03T00:58:40.516ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
75512025-06-03T00:58:40.516ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
75522025-06-03T00:58:40.517ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
75532025-06-03T00:58:40.517ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
75542025-06-03T00:58:40.517ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
75552025-06-03T00:58:40.517ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
75562025-06-03T00:58:40.519ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
75572025-06-03T00:58:40.519ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
75582025-06-03T00:58:40.520ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
75592025-06-03T00:58:40.520ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-afe89ed7-19b7-479b-81eb-05c988ebc66d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-afe89ed7-19b7-479b-81eb-05c988ebc66d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 2847, tv_nsec: 747767639 } })
75602025-06-03T00:58:40.521ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1179
75612025-06-03T00:58:40.521ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
75622025-06-03T00:58:40.521ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
75632025-06-03T00:58:40.521ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
75642025-06-03T00:58:40.525ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
75652025-06-03T00:58:40.525ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
75662025-06-03T00:58:40.527ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-afe89ed7-19b7-479b-81eb-05c988ebc66d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-afe89ed7-19b7-479b-81eb-05c988ebc66d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 2847, tv_nsec: 754237320 } })
75672025-06-03T00:58:40.527ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1179 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae
75682025-06-03T00:58:40.533ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
75692025-06-03T00:58:40.533ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
75702025-06-03T00:58:40.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
75712025-06-03T00:58:40.535ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
75722025-06-03T00:58:40.535ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
75732025-06-03T00:58:40.537ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 4ec93699-353f-4b9e-ae06-25355fdb069e, lookup_type: ById(4ec93699-353f-4b9e-ae06-25355fdb069e) } result = Ok(())
75742025-06-03T00:58:40.537ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
75752025-06-03T00:58:40.537ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
75762025-06-03T00:58:40.552ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 4ec93699-353f-4b9e-ae06-25355fdb069e original_time_created = 2025-06-03 00:51:07.059326 UTC target_id = 4ec93699-353f-4b9e-ae06-25355fdb069e time_created = 2025-06-03 00:51:07.059326 UTC
75772025-06-03T00:58:40.552ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 49.017292ms iteration = 46 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"4ec93699-353f-4b9e-ae06-25355fdb069e","time_created":"2025-06-03T00:51:07.059326Z"}
75782025-06-03T00:58:40.559ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
75792025-06-03T00:58:40.561ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
75802025-06-03T00:58:40.561ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
75812025-06-03T00:58:40.561ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
75822025-06-03T00:58:40.561ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 57.870248ms iteration = 16 status = {"errors":[],"finish_invoked_ok":[]}
75832025-06-03T00:58:40.568ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
75842025-06-03T00:58:40.570ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
75852025-06-03T00:58:40.570ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
75862025-06-03T00:58:40.570ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
75872025-06-03T00:58:40.571ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
75882025-06-03T00:58:40.571ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
75892025-06-03T00:58:40.571ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
75902025-06-03T00:58:40.572ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
75912025-06-03T00:58:40.573ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 69.769447ms iteration = 16 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
75922025-06-03T00:58:40.574ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
75932025-06-03T00:58:40.575ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
75942025-06-03T00:58:40.576ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
75952025-06-03T00:58:40.577ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
75962025-06-03T00:58:40.578ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
75972025-06-03T00:58:40.579ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
75982025-06-03T00:58:40.579ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
75992025-06-03T00:58:40.579ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
76002025-06-03T00:58:40.580ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
76012025-06-03T00:58:40.582ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: afe89ed7-19b7-479b-81eb-05c988ebc66d, lookup_type: ById(afe89ed7-19b7-479b-81eb-05c988ebc66d) } result = Ok(())
76022025-06-03T00:58:40.582ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
76032025-06-03T00:58:40.583ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
76042025-06-03T00:58:40.583ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 79.769117ms iteration = 46 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
76052025-06-03T00:58:40.584ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 81.051987ms iteration = 16 status = 0
76062025-06-03T00:58:40.586ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 82.183412ms iteration = 16 status = {"errors":[],"requests_created_ok":[]}
76072025-06-03T00:58:40.586ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
76082025-06-03T00:58:40.586ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
76092025-06-03T00:58:40.588ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
76102025-06-03T00:58:40.588ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 85.121803ms iteration = 16 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
76112025-06-03T00:58:40.588ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
76122025-06-03T00:58:40.588ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
76132025-06-03T00:58:40.588ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
76142025-06-03T00:58:40.589ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
76152025-06-03T00:58:40.590ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1187 zone_address = fd00:1122:3344:101::2
76162025-06-03T00:58:40.590ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
76172025-06-03T00:58:40.590ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "5e9b1cf9-c549-4717-ba7c-5c1095634167", "content-length": "26", "date": "Tue, 03 Jun 2025 00:58:40 GMT"} })
76182025-06-03T00:58:40.590ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1190 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
76192025-06-03T00:58:40.591ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
76202025-06-03T00:58:40.591ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
76212025-06-03T00:58:40.591ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
76222025-06-03T00:58:40.591ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1224 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
76232025-06-03T00:58:40.591ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
76242025-06-03T00:58:40.591ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
76252025-06-03T00:58:40.592ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
76262025-06-03T00:58:40.592ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
76272025-06-03T00:58:40.592ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
76282025-06-03T00:58:40.593ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
76292025-06-03T00:58:40.594ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
76302025-06-03T00:58:40.596ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
76312025-06-03T00:58:40.596ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
76322025-06-03T00:58:40.596ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
76332025-06-03T00:58:40.613ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:547 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
76342025-06-03T00:58:40.613ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 109.548221ms iteration = 16 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
76352025-06-03T00:58:40.613ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
76362025-06-03T00:58:40.614ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
76372025-06-03T00:58:40.615ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
76382025-06-03T00:58:40.616ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
76392025-06-03T00:58:40.616ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
76402025-06-03T00:58:40.616ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
76412025-06-03T00:58:40.632ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
76422025-06-03T00:58:40.632ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
76432025-06-03T00:58:40.632ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 128.707346ms iteration = 16 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
76442025-06-03T00:58:40.638ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:372
76452025-06-03T00:58:40.638ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 134.565279ms iteration = 16 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
76462025-06-03T00:58:40.660ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:1025 local_addr = [fd00:1122:3344:101::a]:12221 remote_addr = [fd00:1122:3344:101::d]:34094
76472025-06-03T00:58:40.660ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:34094 req_id = cca68bf1-e754-4ef7-a976-6284342bba44 roles = RoleSet { roles: {} } uri = /metrics/collectors/292a19e1-4305-4e49-b11e-08ac74714bf3/producers?sort_by=id_ascending
76482025-06-03T00:58:40.660ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:34094 req_id = cca68bf1-e754-4ef7-a976-6284342bba44 resource = Database result = Ok(()) uri = /metrics/collectors/292a19e1-4305-4e49-b11e-08ac74714bf3/producers?sort_by=id_ascending
76492025-06-03T00:58:40.663ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 160.095551ms iteration = 16 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
76502025-06-03T00:58:40.663ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 3804 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:34094 req_id = cca68bf1-e754-4ef7-a976-6284342bba44 response_code = 200 uri = /metrics/collectors/292a19e1-4305-4e49-b11e-08ac74714bf3/producers?sort_by=id_ascending
76512025-06-03T00:58:40.664ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:34094 req_id = 404d7880-eee8-40aa-9ad1-e4cd45fcd122 roles = RoleSet { roles: {} } uri = /metrics/collectors/292a19e1-4305-4e49-b11e-08ac74714bf3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmZTEyZmZjMS1mMmFjLTRiZjMtOTY3Mi1mYzE4NWY4MmM1OWIifX0%3D
76522025-06-03T00:58:40.664ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:34094 req_id = 404d7880-eee8-40aa-9ad1-e4cd45fcd122 resource = Database result = Ok(()) uri = /metrics/collectors/292a19e1-4305-4e49-b11e-08ac74714bf3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmZTEyZmZjMS1mMmFjLTRiZjMtOTY3Mi1mYzE4NWY4MmM1OWIifX0%3D
76532025-06-03T00:58:40.664ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
76542025-06-03T00:58:40.665ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
76552025-06-03T00:58:40.666ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2215 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:34094 req_id = 404d7880-eee8-40aa-9ad1-e4cd45fcd122 response_code = 200 uri = /metrics/collectors/292a19e1-4305-4e49-b11e-08ac74714bf3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmZTEyZmZjMS1mMmFjLTRiZjMtOTY3Mi1mYzE4NWY4MmM1OWIifX0%3D
76562025-06-03T00:58:40.666ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
76572025-06-03T00:58:40.668ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
76582025-06-03T00:58:40.668ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
76592025-06-03T00:58:40.668ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
76602025-06-03T00:58:40.669ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
76612025-06-03T00:58:40.671ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
76622025-06-03T00:58:40.671ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
76632025-06-03T00:58:40.671ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
76642025-06-03T00:58:40.683ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
76652025-06-03T00:58:40.683ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
76662025-06-03T00:58:40.685ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 181.797256ms iteration = 16 status = null
76672025-06-03T00:58:40.691ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 188.106071ms iteration = 17 status = {"physical_disks_added":0}
76682025-06-03T00:58:40.693ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1187 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae zone_address = fd00:1122:3344:101::2
76692025-06-03T00:58:40.693ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
76702025-06-03T00:58:40.693ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "87bf9cce-881d-4f07-a974-f8a1fcefee5c", "content-length": "26", "date": "Tue, 03 Jun 2025 00:58:40 GMT"} })
76712025-06-03T00:58:40.693ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1190 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
76722025-06-03T00:58:40.695ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1224 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae
76732025-06-03T00:58:40.702ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1187 zone_address = fd00:1122:3344:101::2
76742025-06-03T00:58:40.702ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
76752025-06-03T00:58:40.702ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "a2600308-79e9-48a0-91e6-0ca67a42ba30", "content-length": "26", "date": "Tue, 03 Jun 2025 00:58:40 GMT"} })
76762025-06-03T00:58:40.702ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1190 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
76772025-06-03T00:58:40.704ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1224 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
76782025-06-03T00:58:40.771ZDEBGnexus (BackgroundTasks): client request SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
76792025-06-03T00:58:40.856ZDEBGnexus (BackgroundTasks): client response SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "89005d71-395a-4072-939d-235307304e98", "content-length": "691", "date": "Tue, 03 Jun 2025 00:58:40 GMT"} })
76802025-06-03T00:58:40.856ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
76812025-06-03T00:58:40.856ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
76822025-06-03T00:58:40.856ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
76832025-06-03T00:58:40.857ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 353.770625ms iteration = 16 status = {}
76842025-06-03T00:58:40.866ZINFOnexus (BackgroundTasks): VPC route manager sled afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
76852025-06-03T00:58:40.866ZDEBGnexus (BackgroundTasks): client request SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
76862025-06-03T00:58:40.866ZDEBGnexus (BackgroundTasks): client response SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "9ee27c79-e95d-4646-b8c3-40549ee61fe6", "content-length": "372", "date": "Tue, 03 Jun 2025 00:58:40 GMT"} })
76872025-06-03T00:58:40.866ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
76882025-06-03T00:58:40.866ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
76892025-06-03T00:58:40.882ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 378.559955ms iteration = 16 status = {}
76902025-06-03T00:58:40.906ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
76912025-06-03T00:58:40.906ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
76922025-06-03T00:58:40.909ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: ec5e33f9-c697-46e2-a93e-a745336bf9dc, rack_id: d6831ba0-55f0-4191-8dc2-ea15e55ab0ae, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(9378a284-0ee9-48e2-80a0-3bcaadb1c4e2) } rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae switch_location = Switch0
76932025-06-03T00:58:40.909ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
76942025-06-03T00:58:40.909ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
76952025-06-03T00:58:40.917ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
76962025-06-03T00:58:40.917ZDEBGnexus (BackgroundTasks): client request SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
76972025-06-03T00:58:40.918ZDEBGnexus (BackgroundTasks): client response SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "b9548f3b-cfba-4d57-8376-611647802b0c", "date": "Tue, 03 Jun 2025 00:58:40 GMT"} })
76982025-06-03T00:58:40.918ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
76992025-06-03T00:58:40.918ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
77002025-06-03T00:58:40.919ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
77012025-06-03T00:58:40.919ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
77022025-06-03T00:58:40.921ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1832 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
77032025-06-03T00:58:40.921ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1859 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 9378a284-0ee9-48e2-80a0-3bcaadb1c4e2
77042025-06-03T00:58:40.921ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
77052025-06-03T00:58:40.921ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
77062025-06-03T00:58:40.921ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
77072025-06-03T00:58:40.921ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
77082025-06-03T00:58:40.921ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
77092025-06-03T00:58:40.921ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2006 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
77102025-06-03T00:58:40.921ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:408 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
77112025-06-03T00:58:40.921ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2042 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
77122025-06-03T00:58:40.921ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae
77132025-06-03T00:58:40.921ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
77142025-06-03T00:58:40.921ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
77152025-06-03T00:58:40.923ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
77162025-06-03T00:58:40.923ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
77172025-06-03T00:58:40.923ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae switch_location = Switch0
77182025-06-03T00:58:40.923ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
77192025-06-03T00:58:40.926ZINFOnexus (BackgroundTasks): VPC route manager sled afe89ed7-19b7-479b-81eb-05c988ebc66d push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
77202025-06-03T00:58:40.926ZDEBGnexus (BackgroundTasks): client request SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
77212025-06-03T00:58:41.026ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "38b02ecf-6096-4002-9866-ffafe33fe641", "date": "Tue, 03 Jun 2025 00:58:40 GMT"} })
77222025-06-03T00:58:41.026ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae switch = Switch0
77232025-06-03T00:58:41.026ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
77242025-06-03T00:58:41.026ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "75c554f8-ff30-425d-be46-e1d671bfb4f4", "content-length": "682", "date": "Tue, 03 Jun 2025 00:58:40 GMT"} })
77252025-06-03T00:58:41.026ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
77262025-06-03T00:58:41.027ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
77272025-06-03T00:58:41.027ZDEBGnexus (BackgroundTasks): client response SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "6aeb1dc7-6192-4685-82d0-0216c67706ea", "date": "Tue, 03 Jun 2025 00:58:40 GMT"} })
77282025-06-03T00:58:41.028ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 524.735384ms iteration = 16 status = {}
77292025-06-03T00:58:41.031ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
77302025-06-03T00:58:41.031ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
77312025-06-03T00:58:41.035ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
77322025-06-03T00:58:41.035ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
77332025-06-03T00:58:41.037ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
77342025-06-03T00:58:41.037ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
77352025-06-03T00:58:41.045ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 541.345979ms iteration = 16 status = {}
77362025-06-03T00:58:45.670ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 253 local_addr = [fd00:1122:3344:101::a]:63576 method = GET remote_addr = [fd00:1122:3344:101::d]:61315 req_id = 1c112d0a-8695-40a2-a48d-27ede8918931 response_code = 200 uri = /a2861392-5fd2-462b-84e5-6f36c716263a
77372025-06-03T00:58:50.502ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 47 reason = Timeout
77382025-06-03T00:58:50.503ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
77392025-06-03T00:58:50.503ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 47 reason = Timeout
77402025-06-03T00:58:50.503ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
77412025-06-03T00:58:50.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
77422025-06-03T00:58:50.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
77432025-06-03T00:58:50.504ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
77442025-06-03T00:58:50.507ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
77452025-06-03T00:58:50.507ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
77462025-06-03T00:58:50.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
77472025-06-03T00:58:50.508ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
77482025-06-03T00:58:50.509ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
77492025-06-03T00:58:50.510ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
77502025-06-03T00:58:50.512ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 4ec93699-353f-4b9e-ae06-25355fdb069e, lookup_type: ById(4ec93699-353f-4b9e-ae06-25355fdb069e) } result = Ok(())
77512025-06-03T00:58:50.512ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
77522025-06-03T00:58:50.512ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
77532025-06-03T00:58:50.523ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 4ec93699-353f-4b9e-ae06-25355fdb069e original_time_created = 2025-06-03 00:51:07.059326 UTC target_id = 4ec93699-353f-4b9e-ae06-25355fdb069e time_created = 2025-06-03 00:51:07.059326 UTC
77542025-06-03T00:58:50.523ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 20.916589ms iteration = 47 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"4ec93699-353f-4b9e-ae06-25355fdb069e","time_created":"2025-06-03T00:51:07.059326Z"}
77552025-06-03T00:58:50.534ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
77562025-06-03T00:58:50.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
77572025-06-03T00:58:50.536ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 33.313524ms iteration = 47 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
77582025-06-03T00:58:55.671ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 302 local_addr = [fd00:1122:3344:101::a]:63576 method = GET remote_addr = [fd00:1122:3344:101::d]:61315 req_id = 6105c60c-60ba-4a58-9c59-db0f8eb43790 response_code = 200 uri = /a2861392-5fd2-462b-84e5-6f36c716263a
77592025-06-03T00:59:00.503ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 48 reason = Timeout
77602025-06-03T00:59:00.503ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
77612025-06-03T00:59:00.503ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 48 reason = Timeout
77622025-06-03T00:59:00.503ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
77632025-06-03T00:59:00.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
77642025-06-03T00:59:00.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
77652025-06-03T00:59:00.505ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
77662025-06-03T00:59:00.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
77672025-06-03T00:59:00.505ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
77682025-06-03T00:59:00.508ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
77692025-06-03T00:59:00.508ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
77702025-06-03T00:59:00.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
77712025-06-03T00:59:00.510ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
77722025-06-03T00:59:00.510ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
77732025-06-03T00:59:00.511ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
77742025-06-03T00:59:00.513ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 4ec93699-353f-4b9e-ae06-25355fdb069e, lookup_type: ById(4ec93699-353f-4b9e-ae06-25355fdb069e) } result = Ok(())
77752025-06-03T00:59:00.513ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
77762025-06-03T00:59:00.513ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
77772025-06-03T00:59:00.526ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 4ec93699-353f-4b9e-ae06-25355fdb069e original_time_created = 2025-06-03 00:51:07.059326 UTC target_id = 4ec93699-353f-4b9e-ae06-25355fdb069e time_created = 2025-06-03 00:51:07.059326 UTC
77782025-06-03T00:59:00.526ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 23.539271ms iteration = 48 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"4ec93699-353f-4b9e-ae06-25355fdb069e","time_created":"2025-06-03T00:51:07.059326Z"}
77792025-06-03T00:59:00.539ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 35.76568ms iteration = 48 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
77802025-06-03T00:59:05.670ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 264 local_addr = [fd00:1122:3344:101::a]:63576 method = GET remote_addr = [fd00:1122:3344:101::d]:61315 req_id = c7880f87-1ce5-43ff-831a-d5a16a3b2252 response_code = 200 uri = /a2861392-5fd2-462b-84e5-6f36c716263a
77812025-06-03T00:59:10.502ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 9 reason = Timeout
77822025-06-03T00:59:10.502ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
77832025-06-03T00:59:10.502ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
77842025-06-03T00:59:10.502ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
77852025-06-03T00:59:10.502ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 49 reason = Timeout
77862025-06-03T00:59:10.502ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
77872025-06-03T00:59:10.502ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 17 reason = Timeout
77882025-06-03T00:59:10.502ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 10 reason = Timeout
77892025-06-03T00:59:10.502ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:85 reason = blueprint disabled target_id = 4ec93699-353f-4b9e-ae06-25355fdb069e
77902025-06-03T00:59:10.502ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 10.74\u{b5}s iteration = 10 status = {"enabled":false,"target_id":"4ec93699-353f-4b9e-ae06-25355fdb069e"}
77912025-06-03T00:59:10.502ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 9 reason = Timeout
77922025-06-03T00:59:10.502ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 17 reason = Timeout
77932025-06-03T00:59:10.502ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 17 reason = Timeout
77942025-06-03T00:59:10.502ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
77952025-06-03T00:59:10.502ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
77962025-06-03T00:59:10.502ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 9 reason = Timeout
77972025-06-03T00:59:10.502ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:53
77982025-06-03T00:59:10.502ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 17 reason = Timeout
77992025-06-03T00:59:10.502ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
78002025-06-03T00:59:10.502ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
78012025-06-03T00:59:10.502ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 17 reason = Timeout
78022025-06-03T00:59:10.503ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
78032025-06-03T00:59:10.503ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 9 reason = Timeout
78042025-06-03T00:59:10.503ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
78052025-06-03T00:59:10.503ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
78062025-06-03T00:59:10.503ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 17 reason = Timeout
78072025-06-03T00:59:10.503ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
78082025-06-03T00:59:10.503ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 17 reason = Timeout
78092025-06-03T00:59:10.503ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
78102025-06-03T00:59:10.503ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
78112025-06-03T00:59:10.503ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 17 reason = Timeout
78122025-06-03T00:59:10.503ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 17 reason = Timeout
78132025-06-03T00:59:10.503ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
78142025-06-03T00:59:10.503ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
78152025-06-03T00:59:10.503ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 17 reason = Timeout
78162025-06-03T00:59:10.503ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
78172025-06-03T00:59:10.503ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 17 reason = Timeout
78182025-06-03T00:59:10.503ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
78192025-06-03T00:59:10.503ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
78202025-06-03T00:59:10.503ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 17 reason = Timeout
78212025-06-03T00:59:10.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
78222025-06-03T00:59:10.503ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 17 reason = Timeout
78232025-06-03T00:59:10.503ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
78242025-06-03T00:59:10.503ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
78252025-06-03T00:59:10.503ZDEBGnexus (BackgroundTasks): activating background_task = webhook_deliverator iteration = 9 reason = Timeout
78262025-06-03T00:59:10.503ZDEBGnexus (BackgroundTasks): roles background_task = webhook_deliverator roles = RoleSet { roles: {} }
78272025-06-03T00:59:10.503ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 17 reason = Timeout
78282025-06-03T00:59:10.503ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
78292025-06-03T00:59:10.503ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 49 reason = Timeout
78302025-06-03T00:59:10.503ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 17 reason = Timeout
78312025-06-03T00:59:10.503ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
78322025-06-03T00:59:10.503ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
78332025-06-03T00:59:10.503ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 18 reason = Timeout
78342025-06-03T00:59:10.503ZDEBGnexus (BackgroundTasks): activating background_task = alert_dispatcher iteration = 9 reason = Timeout
78352025-06-03T00:59:10.503ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
78362025-06-03T00:59:10.503ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
78372025-06-03T00:59:10.503ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 17 reason = Timeout
78382025-06-03T00:59:10.503ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
78392025-06-03T00:59:10.503ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 11 reason = Timeout
78402025-06-03T00:59:10.503ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 9 reason = Timeout
78412025-06-03T00:59:10.503ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
78422025-06-03T00:59:10.503ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 11 reason = Timeout
78432025-06-03T00:59:10.503ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 9 reason = Timeout
78442025-06-03T00:59:10.503ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 9 reason = Timeout
78452025-06-03T00:59:10.503ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
78462025-06-03T00:59:10.503ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 9 reason = Timeout
78472025-06-03T00:59:10.503ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
78482025-06-03T00:59:10.503ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 9 reason = Timeout
78492025-06-03T00:59:10.503ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-06-03 00:49:10.503380606 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
78502025-06-03T00:59:10.503ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
78512025-06-03T00:59:10.503ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 9 reason = Timeout
78522025-06-03T00:59:10.503ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
78532025-06-03T00:59:10.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
78542025-06-03T00:59:10.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
78552025-06-03T00:59:10.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
78562025-06-03T00:59:10.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
78572025-06-03T00:59:10.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
78582025-06-03T00:59:10.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
78592025-06-03T00:59:10.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
78602025-06-03T00:59:10.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
78612025-06-03T00:59:10.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
78622025-06-03T00:59:10.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
78632025-06-03T00:59:10.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
78642025-06-03T00:59:10.503ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
78652025-06-03T00:59:10.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
78662025-06-03T00:59:10.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
78672025-06-03T00:59:10.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = webhook_deliverator resource = Database result = Ok(())
78682025-06-03T00:59:10.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
78692025-06-03T00:59:10.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
78702025-06-03T00:59:10.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
78712025-06-03T00:59:10.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
78722025-06-03T00:59:10.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = alert_dispatcher resource = Database result = Ok(())
78732025-06-03T00:59:10.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
78742025-06-03T00:59:10.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
78752025-06-03T00:59:10.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
78762025-06-03T00:59:10.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
78772025-06-03T00:59:10.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
78782025-06-03T00:59:10.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
78792025-06-03T00:59:10.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
78802025-06-03T00:59:10.504ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("241573bc-467c-4802-9f4b-f7c82da4e12e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("64fc0a69-d4b8-4f6c-9a07-77a4cfa94962.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("ceb74144-4b6b-4b3f-bc80-eac2d75f3bc9.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("ceb74144-4b6b-4b3f-bc80-eac2d75f3bc9.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:2::1))) }], valid_until: Instant { tv_sec: 2877, tv_nsec: 731085101 } })
78812025-06-03T00:59:10.504ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("9feb16c9-ff60-43e8-917b-965124f46e0d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("aa124775-31de-4648-981a-4af500268b66.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("aa124775-31de-4648-981a-4af500268b66.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::9))) }], valid_until: Instant { tv_sec: 2877, tv_nsec: 731101282 } })
78822025-06-03T00:59:10.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
78832025-06-03T00:59:10.504ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
78842025-06-03T00:59:10.504ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.92803ms iteration = 9 status = {"addresses":["[fd00:1122:3344:101::8]:5353","[fd00:1122:3344:101::9]:5353"]}
78852025-06-03T00:59:10.504ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } dns_group = internal
78862025-06-03T00:59:10.504ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.514357ms iteration = 9 status = {"addresses":["[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:3::1]:5353","[fd00:1122:3344:2::1]:5353"]}
78872025-06-03T00:59:10.507ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
78882025-06-03T00:59:10.508ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
78892025-06-03T00:59:10.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
78902025-06-03T00:59:10.509ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 6.224584ms iteration = 17 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
78912025-06-03T00:59:10.509ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
78922025-06-03T00:59:10.509ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
78932025-06-03T00:59:10.509ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
78942025-06-03T00:59:10.510ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
78952025-06-03T00:59:10.513ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
78962025-06-03T00:59:10.513ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
78972025-06-03T00:59:10.513ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
78982025-06-03T00:59:10.513ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
78992025-06-03T00:59:10.516ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
79002025-06-03T00:59:10.517ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
79012025-06-03T00:59:10.517ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-afe89ed7-19b7-479b-81eb-05c988ebc66d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-afe89ed7-19b7-479b-81eb-05c988ebc66d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 2877, tv_nsec: 744225549 } })
79022025-06-03T00:59:10.517ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
79032025-06-03T00:59:10.517ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
79042025-06-03T00:59:10.517ZINFOnexus (BackgroundTasks): all webhook delivery tasks completed background_task = webhook_deliverator file = nexus/src/app/background/tasks/webhook_deliverator.rs:202 num_receivers = 0
79052025-06-03T00:59:10.517ZDEBGnexus (BackgroundTasks): activation complete background_task = webhook_deliverator elapsed = 14.377337ms iteration = 9 status = {"by_rx":{},"error":null}
79062025-06-03T00:59:10.517ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-afe89ed7-19b7-479b-81eb-05c988ebc66d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-afe89ed7-19b7-479b-81eb-05c988ebc66d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 2877, tv_nsec: 744604501 } })
79072025-06-03T00:59:10.517ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1179
79082025-06-03T00:59:10.517ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1179
79092025-06-03T00:59:10.520ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
79102025-06-03T00:59:10.520ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
79112025-06-03T00:59:10.520ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
79122025-06-03T00:59:10.525ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 22.011694ms iteration = 17 status = {"errors":[],"garbage_collect_requested":[]}
79132025-06-03T00:59:10.526ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
79142025-06-03T00:59:10.526ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 23.140219ms iteration = 9 status = {"errors":[],"found_port_ok":[]}
79152025-06-03T00:59:10.527ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-06-03 00:49:10.503380606 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
79162025-06-03T00:59:10.527ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 24.280614ms iteration = 9 status = {"expiration":"2025-06-03T00:49:10.503380606Z","pruned":[]}
79172025-06-03T00:59:10.530ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
79182025-06-03T00:59:10.530ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
79192025-06-03T00:59:10.530ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
79202025-06-03T00:59:10.537ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
79212025-06-03T00:59:10.542ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
79222025-06-03T00:59:10.564ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:547 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
79232025-06-03T00:59:10.565ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 62.002166ms iteration = 17 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
79242025-06-03T00:59:10.569ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
79252025-06-03T00:59:10.569ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 66.701721ms iteration = 17 status = {"errors":[],"requests_created_ok":[]}
79262025-06-03T00:59:10.569ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
79272025-06-03T00:59:10.573ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
79282025-06-03T00:59:10.576ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
79292025-06-03T00:59:10.576ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
79302025-06-03T00:59:10.576ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
79312025-06-03T00:59:10.580ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
79322025-06-03T00:59:10.580ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
79332025-06-03T00:59:10.581ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = alert_dispatcher resource = Database result = Ok(())
79342025-06-03T00:59:10.581ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
79352025-06-03T00:59:10.582ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 79.53441ms iteration = 49 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
79362025-06-03T00:59:10.586ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
79372025-06-03T00:59:10.586ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
79382025-06-03T00:59:10.587ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
79392025-06-03T00:59:10.587ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
79402025-06-03T00:59:10.587ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
79412025-06-03T00:59:10.587ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
79422025-06-03T00:59:10.587ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
79432025-06-03T00:59:10.587ZDEBGnexus (BackgroundTasks): activation complete background_task = alert_dispatcher elapsed = 84.627568ms iteration = 9 status = {"dispatched":[],"errors":[],"glob_version":"144.0.0","globs_reprocessed":{},"no_receivers":[]}
79442025-06-03T00:59:10.590ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
79452025-06-03T00:59:10.590ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
79462025-06-03T00:59:10.590ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
79472025-06-03T00:59:10.590ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
79482025-06-03T00:59:10.593ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
79492025-06-03T00:59:10.593ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
79502025-06-03T00:59:10.593ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
79512025-06-03T00:59:10.594ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
79522025-06-03T00:59:10.594ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
79532025-06-03T00:59:10.594ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
79542025-06-03T00:59:10.594ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
79552025-06-03T00:59:10.595ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
79562025-06-03T00:59:10.595ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 92.387029ms iteration = 17 status = {"errors":[],"finish_invoked_ok":[]}
79572025-06-03T00:59:10.596ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
79582025-06-03T00:59:10.596ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
79592025-06-03T00:59:10.596ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
79602025-06-03T00:59:10.598ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
79612025-06-03T00:59:10.600ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
79622025-06-03T00:59:10.600ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
79632025-06-03T00:59:10.600ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
79642025-06-03T00:59:10.600ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
79652025-06-03T00:59:10.600ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
79662025-06-03T00:59:10.600ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
79672025-06-03T00:59:10.600ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
79682025-06-03T00:59:10.600ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
79692025-06-03T00:59:10.601ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
79702025-06-03T00:59:10.601ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
79712025-06-03T00:59:10.605ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
79722025-06-03T00:59:10.606ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
79732025-06-03T00:59:10.606ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:167 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
79742025-06-03T00:59:10.607ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 103.617498ms iteration = 9 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
79752025-06-03T00:59:10.609ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
79762025-06-03T00:59:10.610ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
79772025-06-03T00:59:10.611ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
79782025-06-03T00:59:10.629ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
79792025-06-03T00:59:10.633ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
79802025-06-03T00:59:10.633ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:137 found = 0 not_ready_to_be_deleted = 0
79812025-06-03T00:59:10.633ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 130.694148ms iteration = 9 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
79822025-06-03T00:59:10.646ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
79832025-06-03T00:59:10.660ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:34094 req_id = d987de6b-4ba5-4ed7-8d1b-3f1d232d595d roles = RoleSet { roles: {} } uri = /metrics/collectors/292a19e1-4305-4e49-b11e-08ac74714bf3/producers?sort_by=id_ascending
79842025-06-03T00:59:10.660ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:34094 req_id = d987de6b-4ba5-4ed7-8d1b-3f1d232d595d resource = Database result = Ok(()) uri = /metrics/collectors/292a19e1-4305-4e49-b11e-08ac74714bf3/producers?sort_by=id_ascending
79852025-06-03T00:59:10.662ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
79862025-06-03T00:59:10.662ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
79872025-06-03T00:59:10.662ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
79882025-06-03T00:59:10.662ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
79892025-06-03T00:59:10.663ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
79902025-06-03T00:59:10.676ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1187 zone_address = fd00:1122:3344:101::2
79912025-06-03T00:59:10.676ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
79922025-06-03T00:59:10.676ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "007970f6-1440-4396-99e6-7ecea018976c", "content-length": "26", "date": "Tue, 03 Jun 2025 00:59:10 GMT"} })
79932025-06-03T00:59:10.676ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1190 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
79942025-06-03T00:59:10.677ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
79952025-06-03T00:59:10.677ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
79962025-06-03T00:59:10.678ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1224 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
79972025-06-03T00:59:10.679ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1187 zone_address = fd00:1122:3344:101::2
79982025-06-03T00:59:10.679ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
79992025-06-03T00:59:10.679ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "f56c44a0-af8a-4d41-8c3f-ed050e59a1ab", "content-length": "26", "date": "Tue, 03 Jun 2025 00:59:10 GMT"} })
80002025-06-03T00:59:10.679ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1190 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
80012025-06-03T00:59:10.681ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1224 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
80022025-06-03T00:59:10.716ZINFOnexus (BackgroundTasks): VPC route manager sled afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
80032025-06-03T00:59:10.716ZDEBGnexus (BackgroundTasks): client request SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
80042025-06-03T00:59:10.717ZDEBGnexus (BackgroundTasks): client response SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "30dd16d4-edad-4f3c-9290-a589eae92655", "content-length": "372", "date": "Tue, 03 Jun 2025 00:59:10 GMT"} })
80052025-06-03T00:59:10.717ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
80062025-06-03T00:59:10.717ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
80072025-06-03T00:59:10.718ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
80082025-06-03T00:59:10.721ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
80092025-06-03T00:59:10.721ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
80102025-06-03T00:59:10.721ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
80112025-06-03T00:59:10.721ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
80122025-06-03T00:59:10.721ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
80132025-06-03T00:59:10.721ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
80142025-06-03T00:59:10.722ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
80152025-06-03T00:59:10.722ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
80162025-06-03T00:59:10.722ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 4ec93699-353f-4b9e-ae06-25355fdb069e, lookup_type: ById(4ec93699-353f-4b9e-ae06-25355fdb069e) } result = Ok(())
80172025-06-03T00:59:10.722ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
80182025-06-03T00:59:10.723ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
80192025-06-03T00:59:10.725ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
80202025-06-03T00:59:10.726ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
80212025-06-03T00:59:10.728ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 68334 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:34094 req_id = d987de6b-4ba5-4ed7-8d1b-3f1d232d595d response_code = 200 uri = /metrics/collectors/292a19e1-4305-4e49-b11e-08ac74714bf3/producers?sort_by=id_ascending
80222025-06-03T00:59:10.728ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-afe89ed7-19b7-479b-81eb-05c988ebc66d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-afe89ed7-19b7-479b-81eb-05c988ebc66d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 2877, tv_nsec: 955403916 } })
80232025-06-03T00:59:10.728ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:34094 req_id = 227526a3-1a52-42de-a6b4-b3012307daae roles = RoleSet { roles: {} } uri = /metrics/collectors/292a19e1-4305-4e49-b11e-08ac74714bf3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmZTEyZmZjMS1mMmFjLTRiZjMtOTY3Mi1mYzE4NWY4MmM1OWIifX0%3D
80242025-06-03T00:59:10.728ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1179 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae
80252025-06-03T00:59:10.728ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:34094 req_id = 227526a3-1a52-42de-a6b4-b3012307daae resource = Database result = Ok(()) uri = /metrics/collectors/292a19e1-4305-4e49-b11e-08ac74714bf3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmZTEyZmZjMS1mMmFjLTRiZjMtOTY3Mi1mYzE4NWY4MmM1OWIifX0%3D
80262025-06-03T00:59:10.744ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 15884 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:34094 req_id = 227526a3-1a52-42de-a6b4-b3012307daae response_code = 200 uri = /metrics/collectors/292a19e1-4305-4e49-b11e-08ac74714bf3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmZTEyZmZjMS1mMmFjLTRiZjMtOTY3Mi1mYzE4NWY4MmM1OWIifX0%3D
80272025-06-03T00:59:10.757ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 4ec93699-353f-4b9e-ae06-25355fdb069e original_time_created = 2025-06-03 00:51:07.059326 UTC target_id = 4ec93699-353f-4b9e-ae06-25355fdb069e time_created = 2025-06-03 00:51:07.059326 UTC
80282025-06-03T00:59:10.757ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 254.131221ms iteration = 49 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"4ec93699-353f-4b9e-ae06-25355fdb069e","time_created":"2025-06-03T00:51:07.059326Z"}
80292025-06-03T00:59:10.769ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
80302025-06-03T00:59:10.769ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "f13af143-0c08-46d4-9d22-d38d65ac7e7b", "date": "Tue, 03 Jun 2025 00:59:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
80312025-06-03T00:59:10.769ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
80322025-06-03T00:59:10.771ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "dc3c42ac-d004-43d0-b973-69762716c7c2", "date": "Tue, 03 Jun 2025 00:59:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
80332025-06-03T00:59:10.771ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
80342025-06-03T00:59:10.773ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
80352025-06-03T00:59:10.773ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 270.731446ms iteration = 11 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
80362025-06-03T00:59:10.775ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
80372025-06-03T00:59:10.775ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
80382025-06-03T00:59:10.776ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
80392025-06-03T00:59:10.777ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
80402025-06-03T00:59:10.777ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
80412025-06-03T00:59:10.778ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
80422025-06-03T00:59:10.779ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
80432025-06-03T00:59:10.779ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 276.421283ms iteration = 17 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
80442025-06-03T00:59:10.779ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
80452025-06-03T00:59:10.780ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 277.138456ms iteration = 17 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
80462025-06-03T00:59:10.780ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
80472025-06-03T00:59:10.781ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
80482025-06-03T00:59:10.782ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
80492025-06-03T00:59:10.782ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
80502025-06-03T00:59:10.782ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
80512025-06-03T00:59:10.784ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
80522025-06-03T00:59:10.786ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: afe89ed7-19b7-479b-81eb-05c988ebc66d, lookup_type: ById(afe89ed7-19b7-479b-81eb-05c988ebc66d) } result = Ok(())
80532025-06-03T00:59:10.786ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
80542025-06-03T00:59:10.786ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
80552025-06-03T00:59:10.787ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
80562025-06-03T00:59:10.787ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
80572025-06-03T00:59:10.788ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
80582025-06-03T00:59:10.788ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 285.512686ms iteration = 17 status = 0
80592025-06-03T00:59:10.789ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
80602025-06-03T00:59:10.789ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
80612025-06-03T00:59:10.789ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
80622025-06-03T00:59:10.790ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
80632025-06-03T00:59:10.794ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
80642025-06-03T00:59:10.794ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
80652025-06-03T00:59:10.794ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
80662025-06-03T00:59:10.795ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
80672025-06-03T00:59:10.795ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
80682025-06-03T00:59:10.795ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 292.72891ms iteration = 17 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
80692025-06-03T00:59:10.796ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
80702025-06-03T00:59:10.796ZDEBGnexus (BackgroundTasks): client request SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
80712025-06-03T00:59:10.796ZDEBGnexus (BackgroundTasks): client response SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "9524ba96-7928-4f2d-a84e-85d1f5b0f104", "date": "Tue, 03 Jun 2025 00:59:10 GMT"} })
80722025-06-03T00:59:10.796ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
80732025-06-03T00:59:10.796ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
80742025-06-03T00:59:10.797ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
80752025-06-03T00:59:10.797ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
80762025-06-03T00:59:10.799ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
80772025-06-03T00:59:10.799ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
80782025-06-03T00:59:10.801ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
80792025-06-03T00:59:10.801ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
80802025-06-03T00:59:10.802ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
80812025-06-03T00:59:10.802ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
80822025-06-03T00:59:10.802ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
80832025-06-03T00:59:10.802ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
80842025-06-03T00:59:10.806ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
80852025-06-03T00:59:10.806ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
80862025-06-03T00:59:10.810ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
80872025-06-03T00:59:10.810ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
80882025-06-03T00:59:10.815ZINFOnexus (BackgroundTasks): VPC route manager sled afe89ed7-19b7-479b-81eb-05c988ebc66d push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
80892025-06-03T00:59:10.815ZDEBGnexus (BackgroundTasks): client request SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
80902025-06-03T00:59:10.815ZDEBGnexus (BackgroundTasks): client response SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "5ac5bb5f-7714-4b8b-b7e8-a15b92728bfb", "date": "Tue, 03 Jun 2025 00:59:10 GMT"} })
80912025-06-03T00:59:10.819ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
80922025-06-03T00:59:10.819ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
80932025-06-03T00:59:10.820ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 317.416706ms iteration = 17 status = {}
80942025-06-03T00:59:10.822ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 318.982914ms iteration = 17 status = null
80952025-06-03T00:59:10.825ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 322.744891ms iteration = 17 status = {}
80962025-06-03T00:59:10.830ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1187 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae zone_address = fd00:1122:3344:101::2
80972025-06-03T00:59:10.830ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
80982025-06-03T00:59:10.830ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "f5d792a8-2012-4fc3-93bd-da8e55f3ed59", "content-length": "26", "date": "Tue, 03 Jun 2025 00:59:10 GMT"} })
80992025-06-03T00:59:10.830ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1190 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
81002025-06-03T00:59:10.837ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:372
81012025-06-03T00:59:10.837ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 334.599279ms iteration = 17 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
81022025-06-03T00:59:10.837ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1224 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae
81032025-06-03T00:59:10.843ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 340.40858ms iteration = 17 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
81042025-06-03T00:59:10.857ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
81052025-06-03T00:59:10.857ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 9e79f9b1-a4de-4f9d-a901-16cc4f79e15d dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
81062025-06-03T00:59:10.857ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
81072025-06-03T00:59:10.857ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
81082025-06-03T00:59:10.859ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
81092025-06-03T00:59:10.860ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
81102025-06-03T00:59:10.860ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 3ef110b4-9400-4c37-8d7f-5552c8a94db0 dns_zone_name = oxide.test found_so_far = 0 version = 2
81112025-06-03T00:59:10.860ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
81122025-06-03T00:59:10.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
81132025-06-03T00:59:10.863ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-06-03T00:51:07.585932Z, time_modified: 2025-06-03T00:51:07.585932Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: ea0ef520-6bf3-4fbd-9ff0-f3698b48b749, db_silo: Silo { identity: SiloIdentity { id: ea0ef520-6bf3-4fbd-9ff0-f3698b48b749, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-06-03T00:51:38.523369Z, time_modified: 2025-06-03T00:51:38.523369Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "bcda8a9f16e63c3cec49f9759478b9f335a661639158b0a9026f69b846d4e127" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: ea0ef520-6bf3-4fbd-9ff0-f3698b48b749, db_silo: Silo { identity: SiloIdentity { id: ea0ef520-6bf3-4fbd-9ff0-f3698b48b749, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-06-03T00:51:38.523369Z, time_modified: 2025-06-03T00:51:38.523369Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "bcda8a9f16e63c3cec49f9759478b9f335a661639158b0a9026f69b846d4e127" }] }) }
81142025-06-03T00:59:10.863ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 360.542996ms iteration = 9 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"ea0ef520-6bf3-4fbd-9ff0-f3698b48b749","tls_certs":["bcda8a9f16e63c3cec49f9759478b9f335a661639158b0a9026f69b846d4e127"]}},"default_endpoint":{"silo_id":"ea0ef520-6bf3-4fbd-9ff0-f3698b48b749","tls_certs":["bcda8a9f16e63c3cec49f9759478b9f335a661639158b0a9026f69b846d4e127"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
81152025-06-03T00:59:10.864ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
81162025-06-03T00:59:10.864ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
81172025-06-03T00:59:10.864ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
81182025-06-03T00:59:10.867ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 9e79f9b1-a4de-4f9d-a901-16cc4f79e15d dns_zone_name = control-plane.oxide.internal found_so_far = 59 version = 1
81192025-06-03T00:59:10.867ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
81202025-06-03T00:59:10.867ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2025-06-03 00:51:38.152773 UTC dns_group = internal generation = 1
81212025-06-03T00:59:10.867ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 364.940412ms iteration = 9 status = {"generation":1}
81222025-06-03T00:59:10.875ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
81232025-06-03T00:59:10.875ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "de9c99c7-64d6-4e93-b6e6-888c65238c93", "date": "Tue, 03 Jun 2025 00:59:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
81242025-06-03T00:59:10.875ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
81252025-06-03T00:59:10.877ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "a293ecc1-9cd2-46d8-afed-ecc16b9adbd9", "date": "Tue, 03 Jun 2025 00:59:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
81262025-06-03T00:59:10.877ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
81272025-06-03T00:59:10.879ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "3daf7ac7-7802-40e3-b604-ddd44b0270fa", "date": "Tue, 03 Jun 2025 00:59:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
81282025-06-03T00:59:10.879ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
81292025-06-03T00:59:10.881ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
81302025-06-03T00:59:10.881ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 378.670228ms iteration = 11 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
81312025-06-03T00:59:10.910ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
81322025-06-03T00:59:10.910ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
81332025-06-03T00:59:10.912ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
81342025-06-03T00:59:10.914ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
81352025-06-03T00:59:10.914ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
81362025-06-03T00:59:10.914ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
81372025-06-03T00:59:10.933ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
81382025-06-03T00:59:10.937ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
81392025-06-03T00:59:10.938ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
81402025-06-03T00:59:10.938ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
81412025-06-03T00:59:10.941ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 3ef110b4-9400-4c37-8d7f-5552c8a94db0 dns_zone_name = oxide.test found_so_far = 4 version = 2
81422025-06-03T00:59:10.941ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
81432025-06-03T00:59:10.941ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2025-06-03 00:51:38.567192 UTC dns_group = external generation = 2
81442025-06-03T00:59:10.941ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 438.836037ms iteration = 9 status = {"generation":2}
81452025-06-03T00:59:10.953ZINFOnexus (BackgroundTasks): instance reincarnation completed successfully background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:113 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0
81462025-06-03T00:59:10.953ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 449.969572ms iteration = 9 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
81472025-06-03T00:59:10.957ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 454.687199ms iteration = 18 status = {"physical_disks_added":0}
81482025-06-03T00:59:10.971ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
81492025-06-03T00:59:10.973ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
81502025-06-03T00:59:10.973ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
81512025-06-03T00:59:10.973ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
81522025-06-03T00:59:10.976ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
81532025-06-03T00:59:10.977ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
81542025-06-03T00:59:10.978ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: ec5e33f9-c697-46e2-a93e-a745336bf9dc, rack_id: d6831ba0-55f0-4191-8dc2-ea15e55ab0ae, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(9378a284-0ee9-48e2-80a0-3bcaadb1c4e2) } rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae switch_location = Switch0
81552025-06-03T00:59:10.978ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
81562025-06-03T00:59:10.978ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
81572025-06-03T00:59:10.991ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1832 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
81582025-06-03T00:59:10.991ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1859 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 9378a284-0ee9-48e2-80a0-3bcaadb1c4e2
81592025-06-03T00:59:10.992ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
81602025-06-03T00:59:10.992ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
81612025-06-03T00:59:10.992ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
81622025-06-03T00:59:10.992ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
81632025-06-03T00:59:10.992ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
81642025-06-03T00:59:10.992ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2006 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
81652025-06-03T00:59:10.992ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:408 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
81662025-06-03T00:59:10.992ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2042 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
81672025-06-03T00:59:10.992ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae
81682025-06-03T00:59:10.992ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
81692025-06-03T00:59:10.992ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
81702025-06-03T00:59:10.994ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae switch_location = Switch0
81712025-06-03T00:59:10.994ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
81722025-06-03T00:59:11.069ZDEBGnexus (BackgroundTasks): client request SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
81732025-06-03T00:59:11.071ZDEBGnexus (BackgroundTasks): client response SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "bb8105d2-d550-4405-a785-38b958d04866", "content-length": "691", "date": "Tue, 03 Jun 2025 00:59:11 GMT"} })
81742025-06-03T00:59:11.071ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
81752025-06-03T00:59:11.071ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
81762025-06-03T00:59:11.071ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
81772025-06-03T00:59:11.072ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 569.294178ms iteration = 17 status = {}
81782025-06-03T00:59:11.210ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "5ba01958-90e5-4866-8c3e-dd51570e0f41", "date": "Tue, 03 Jun 2025 00:59:11 GMT"} })
81792025-06-03T00:59:11.210ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae switch = Switch0
81802025-06-03T00:59:11.210ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
81812025-06-03T00:59:11.211ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "d2a84734-2745-4133-8c11-eeca40fb630c", "content-length": "682", "date": "Tue, 03 Jun 2025 00:59:10 GMT"} })
81822025-06-03T00:59:11.211ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
81832025-06-03T00:59:11.211ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
81842025-06-03T00:59:11.215ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
81852025-06-03T00:59:11.215ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
81862025-06-03T00:59:11.219ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
81872025-06-03T00:59:11.219ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
81882025-06-03T00:59:11.222ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
81892025-06-03T00:59:11.222ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
81902025-06-03T00:59:11.229ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 726.469227ms iteration = 17 status = {}
81912025-06-03T00:59:15.671ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 250 local_addr = [fd00:1122:3344:101::a]:63576 method = GET remote_addr = [fd00:1122:3344:101::d]:61315 req_id = 7ed36858-23f5-4ca9-8d8b-893b8d39bd1e response_code = 200 uri = /a2861392-5fd2-462b-84e5-6f36c716263a
81922025-06-03T00:59:20.502ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 50 reason = Timeout
81932025-06-03T00:59:20.502ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 50 reason = Timeout
81942025-06-03T00:59:20.502ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
81952025-06-03T00:59:20.502ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
81962025-06-03T00:59:20.502ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
81972025-06-03T00:59:20.502ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
81982025-06-03T00:59:20.503ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
81992025-06-03T00:59:20.506ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
82002025-06-03T00:59:20.506ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
82012025-06-03T00:59:20.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
82022025-06-03T00:59:20.507ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
82032025-06-03T00:59:20.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
82042025-06-03T00:59:20.533ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
82052025-06-03T00:59:20.535ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
82062025-06-03T00:59:20.546ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
82072025-06-03T00:59:20.548ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 4ec93699-353f-4b9e-ae06-25355fdb069e, lookup_type: ById(4ec93699-353f-4b9e-ae06-25355fdb069e) } result = Ok(())
82082025-06-03T00:59:20.548ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
82092025-06-03T00:59:20.548ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
82102025-06-03T00:59:20.560ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 4ec93699-353f-4b9e-ae06-25355fdb069e original_time_created = 2025-06-03 00:51:07.059326 UTC target_id = 4ec93699-353f-4b9e-ae06-25355fdb069e time_created = 2025-06-03 00:51:07.059326 UTC
82112025-06-03T00:59:20.560ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 58.30391ms iteration = 50 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"4ec93699-353f-4b9e-ae06-25355fdb069e","time_created":"2025-06-03T00:51:07.059326Z"}
82122025-06-03T00:59:20.567ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 65.05656ms iteration = 50 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
82132025-06-03T00:59:25.670ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 277 local_addr = [fd00:1122:3344:101::a]:63576 method = GET remote_addr = [fd00:1122:3344:101::d]:61315 req_id = ef182cd4-4573-4dc6-8a4e-5ca59b75e177 response_code = 200 uri = /a2861392-5fd2-462b-84e5-6f36c716263a
82142025-06-03T00:59:30.502ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 51 reason = Timeout
82152025-06-03T00:59:30.502ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
82162025-06-03T00:59:30.502ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 51 reason = Timeout
82172025-06-03T00:59:30.502ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
82182025-06-03T00:59:30.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
82192025-06-03T00:59:30.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
82202025-06-03T00:59:30.504ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
82212025-06-03T00:59:30.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
82222025-06-03T00:59:30.505ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
82232025-06-03T00:59:30.507ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
82242025-06-03T00:59:30.507ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
82252025-06-03T00:59:30.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
82262025-06-03T00:59:30.509ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
82272025-06-03T00:59:30.510ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
82282025-06-03T00:59:30.511ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
82292025-06-03T00:59:30.513ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 4ec93699-353f-4b9e-ae06-25355fdb069e, lookup_type: ById(4ec93699-353f-4b9e-ae06-25355fdb069e) } result = Ok(())
82302025-06-03T00:59:30.513ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
82312025-06-03T00:59:30.513ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
82322025-06-03T00:59:30.523ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 4ec93699-353f-4b9e-ae06-25355fdb069e original_time_created = 2025-06-03 00:51:07.059326 UTC target_id = 4ec93699-353f-4b9e-ae06-25355fdb069e time_created = 2025-06-03 00:51:07.059326 UTC
82332025-06-03T00:59:30.523ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 21.196068ms iteration = 51 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"4ec93699-353f-4b9e-ae06-25355fdb069e","time_created":"2025-06-03T00:51:07.059326Z"}
82342025-06-03T00:59:30.536ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 33.721437ms iteration = 51 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
82352025-06-03T00:59:35.671ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 457 local_addr = [fd00:1122:3344:101::a]:63576 method = GET remote_addr = [fd00:1122:3344:101::d]:61315 req_id = cea0ba14-bde2-41b9-a235-7d9ed358db2a response_code = 200 uri = /a2861392-5fd2-462b-84e5-6f36c716263a
82362025-06-03T00:59:40.502ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 52 reason = Timeout
82372025-06-03T00:59:40.502ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
82382025-06-03T00:59:40.502ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 18 reason = Timeout
82392025-06-03T00:59:40.502ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
82402025-06-03T00:59:40.502ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
82412025-06-03T00:59:40.502ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 18 reason = Timeout
82422025-06-03T00:59:40.502ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 18 reason = Timeout
82432025-06-03T00:59:40.502ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
82442025-06-03T00:59:40.502ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
82452025-06-03T00:59:40.502ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 18 reason = Timeout
82462025-06-03T00:59:40.502ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 18 reason = Timeout
82472025-06-03T00:59:40.502ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
82482025-06-03T00:59:40.502ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
82492025-06-03T00:59:40.502ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 18 reason = Timeout
82502025-06-03T00:59:40.502ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
82512025-06-03T00:59:40.502ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 19 reason = Timeout
82522025-06-03T00:59:40.502ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
82532025-06-03T00:59:40.502ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 18 reason = Timeout
82542025-06-03T00:59:40.502ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 18 reason = Timeout
82552025-06-03T00:59:40.502ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
82562025-06-03T00:59:40.502ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
82572025-06-03T00:59:40.502ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 18 reason = Timeout
82582025-06-03T00:59:40.502ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
82592025-06-03T00:59:40.502ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 18 reason = Timeout
82602025-06-03T00:59:40.502ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
82612025-06-03T00:59:40.502ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
82622025-06-03T00:59:40.502ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 18 reason = Timeout
82632025-06-03T00:59:40.502ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 18 reason = Timeout
82642025-06-03T00:59:40.502ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
82652025-06-03T00:59:40.503ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 18 reason = Timeout
82662025-06-03T00:59:40.503ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 18 reason = Timeout
82672025-06-03T00:59:40.503ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 52 reason = Timeout
82682025-06-03T00:59:40.503ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
82692025-06-03T00:59:40.503ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
82702025-06-03T00:59:40.503ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
82712025-06-03T00:59:40.503ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 18 reason = Timeout
82722025-06-03T00:59:40.503ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
82732025-06-03T00:59:40.503ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 18 reason = Timeout
82742025-06-03T00:59:40.503ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
82752025-06-03T00:59:40.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
82762025-06-03T00:59:40.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
82772025-06-03T00:59:40.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
82782025-06-03T00:59:40.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
82792025-06-03T00:59:40.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
82802025-06-03T00:59:40.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
82812025-06-03T00:59:40.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
82822025-06-03T00:59:40.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
82832025-06-03T00:59:40.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
82842025-06-03T00:59:40.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
82852025-06-03T00:59:40.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
82862025-06-03T00:59:40.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
82872025-06-03T00:59:40.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
82882025-06-03T00:59:40.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
82892025-06-03T00:59:40.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
82902025-06-03T00:59:40.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
82912025-06-03T00:59:40.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
82922025-06-03T00:59:40.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
82932025-06-03T00:59:40.505ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
82942025-06-03T00:59:40.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
82952025-06-03T00:59:40.506ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
82962025-06-03T00:59:40.507ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 4.50808ms iteration = 18 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
82972025-06-03T00:59:40.509ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
82982025-06-03T00:59:40.509ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
82992025-06-03T00:59:40.510ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
83002025-06-03T00:59:40.513ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-afe89ed7-19b7-479b-81eb-05c988ebc66d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-afe89ed7-19b7-479b-81eb-05c988ebc66d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 2907, tv_nsec: 740444882 } })
83012025-06-03T00:59:40.513ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1179
83022025-06-03T00:59:40.517ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
83032025-06-03T00:59:40.521ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
83042025-06-03T00:59:40.521ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
83052025-06-03T00:59:40.522ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
83062025-06-03T00:59:40.524ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-afe89ed7-19b7-479b-81eb-05c988ebc66d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-afe89ed7-19b7-479b-81eb-05c988ebc66d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 2907, tv_nsec: 752352621 } })
83072025-06-03T00:59:40.525ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
83082025-06-03T00:59:40.525ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1179
83092025-06-03T00:59:40.528ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
83102025-06-03T00:59:40.528ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
83112025-06-03T00:59:40.528ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
83122025-06-03T00:59:40.528ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-afe89ed7-19b7-479b-81eb-05c988ebc66d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-afe89ed7-19b7-479b-81eb-05c988ebc66d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 2907, tv_nsec: 756325435 } })
83132025-06-03T00:59:40.529ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1179 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae
83142025-06-03T00:59:40.531ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
83152025-06-03T00:59:40.533ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
83162025-06-03T00:59:40.533ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
83172025-06-03T00:59:40.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
83182025-06-03T00:59:40.542ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
83192025-06-03T00:59:40.542ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
83202025-06-03T00:59:40.569ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
83212025-06-03T00:59:40.569ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
83222025-06-03T00:59:40.574ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
83232025-06-03T00:59:40.576ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
83242025-06-03T00:59:40.577ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
83252025-06-03T00:59:40.577ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
83262025-06-03T00:59:40.577ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
83272025-06-03T00:59:40.577ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
83282025-06-03T00:59:40.577ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
83292025-06-03T00:59:40.578ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
83302025-06-03T00:59:40.580ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
83312025-06-03T00:59:40.583ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
83322025-06-03T00:59:40.583ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
83332025-06-03T00:59:40.583ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
83342025-06-03T00:59:40.583ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
83352025-06-03T00:59:40.585ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 82.910444ms iteration = 18 status = {"errors":[],"garbage_collect_requested":[]}
83362025-06-03T00:59:40.586ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
83372025-06-03T00:59:40.586ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
83382025-06-03T00:59:40.586ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
83392025-06-03T00:59:40.592ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
83402025-06-03T00:59:40.593ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
83412025-06-03T00:59:40.593ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
83422025-06-03T00:59:40.593ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
83432025-06-03T00:59:40.593ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 90.830011ms iteration = 18 status = {"errors":[],"finish_invoked_ok":[]}
83442025-06-03T00:59:40.594ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
83452025-06-03T00:59:40.597ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
83462025-06-03T00:59:40.597ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
83472025-06-03T00:59:40.597ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
83482025-06-03T00:59:40.599ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
83492025-06-03T00:59:40.599ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 97.228519ms iteration = 18 status = {"errors":[],"requests_created_ok":[]}
83502025-06-03T00:59:40.600ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
83512025-06-03T00:59:40.604ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
83522025-06-03T00:59:40.604ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
83532025-06-03T00:59:40.604ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
83542025-06-03T00:59:40.607ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 4ec93699-353f-4b9e-ae06-25355fdb069e, lookup_type: ById(4ec93699-353f-4b9e-ae06-25355fdb069e) } result = Ok(())
83552025-06-03T00:59:40.607ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
83562025-06-03T00:59:40.608ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
83572025-06-03T00:59:40.610ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 108.186569ms iteration = 52 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
83582025-06-03T00:59:40.624ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 4ec93699-353f-4b9e-ae06-25355fdb069e original_time_created = 2025-06-03 00:51:07.059326 UTC target_id = 4ec93699-353f-4b9e-ae06-25355fdb069e time_created = 2025-06-03 00:51:07.059326 UTC
83592025-06-03T00:59:40.624ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 121.899995ms iteration = 52 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"4ec93699-353f-4b9e-ae06-25355fdb069e","time_created":"2025-06-03T00:51:07.059326Z"}
83602025-06-03T00:59:40.642ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
83612025-06-03T00:59:40.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
83622025-06-03T00:59:40.643ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1187 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae zone_address = fd00:1122:3344:101::2
83632025-06-03T00:59:40.643ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
83642025-06-03T00:59:40.643ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "abba29ba-ad25-4879-844e-480df26fbbf1", "content-length": "26", "date": "Tue, 03 Jun 2025 00:59:40 GMT"} })
83652025-06-03T00:59:40.643ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1190 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
83662025-06-03T00:59:40.645ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1224 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae
83672025-06-03T00:59:40.648ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
83682025-06-03T00:59:40.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
83692025-06-03T00:59:40.650ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
83702025-06-03T00:59:40.650ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
83712025-06-03T00:59:40.650ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
83722025-06-03T00:59:40.651ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
83732025-06-03T00:59:40.651ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
83742025-06-03T00:59:40.651ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
83752025-06-03T00:59:40.653ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 150.334078ms iteration = 18 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
83762025-06-03T00:59:40.653ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
83772025-06-03T00:59:40.653ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
83782025-06-03T00:59:40.653ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
83792025-06-03T00:59:40.654ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
83802025-06-03T00:59:40.656ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: afe89ed7-19b7-479b-81eb-05c988ebc66d, lookup_type: ById(afe89ed7-19b7-479b-81eb-05c988ebc66d) } result = Ok(())
83812025-06-03T00:59:40.657ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
83822025-06-03T00:59:40.657ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
83832025-06-03T00:59:40.658ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
83842025-06-03T00:59:40.658ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
83852025-06-03T00:59:40.658ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
83862025-06-03T00:59:40.659ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 157.017285ms iteration = 18 status = 0
83872025-06-03T00:59:40.659ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 157.131379ms iteration = 18 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
83882025-06-03T00:59:40.667ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:547 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
83892025-06-03T00:59:40.667ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 164.225179ms iteration = 18 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
83902025-06-03T00:59:40.671ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1187 zone_address = fd00:1122:3344:101::2
83912025-06-03T00:59:40.671ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
83922025-06-03T00:59:40.672ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "08a00df3-eaea-42d5-a649-6202f4943ecd", "content-length": "26", "date": "Tue, 03 Jun 2025 00:59:40 GMT"} })
83932025-06-03T00:59:40.672ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1190 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
83942025-06-03T00:59:40.675ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1224 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
83952025-06-03T00:59:40.693ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1187 zone_address = fd00:1122:3344:101::2
83962025-06-03T00:59:40.693ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
83972025-06-03T00:59:40.693ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "0e637763-b82f-45fb-a336-b8356ec0e1ae", "content-length": "26", "date": "Tue, 03 Jun 2025 00:59:40 GMT"} })
83982025-06-03T00:59:40.693ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1190 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
83992025-06-03T00:59:40.693ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
84002025-06-03T00:59:40.695ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1224 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
84012025-06-03T00:59:40.696ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
84022025-06-03T00:59:40.696ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
84032025-06-03T00:59:40.696ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
84042025-06-03T00:59:40.706ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
84052025-06-03T00:59:40.706ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
84062025-06-03T00:59:40.706ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 203.472098ms iteration = 18 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
84072025-06-03T00:59:40.717ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
84082025-06-03T00:59:40.717ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 214.945664ms iteration = 18 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
84092025-06-03T00:59:40.717ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
84102025-06-03T00:59:40.749ZINFOnexus (BackgroundTasks): VPC route manager sled afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
84112025-06-03T00:59:40.749ZDEBGnexus (BackgroundTasks): client request SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
84122025-06-03T00:59:40.767ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
84132025-06-03T00:59:40.767ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
84142025-06-03T00:59:40.769ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 266.611789ms iteration = 18 status = null
84152025-06-03T00:59:40.778ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 275.714782ms iteration = 18 status = {}
84162025-06-03T00:59:40.796ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:372
84172025-06-03T00:59:40.796ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 293.968488ms iteration = 18 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
84182025-06-03T00:59:40.807ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 304.414963ms iteration = 19 status = {"physical_disks_added":0}
84192025-06-03T00:59:40.846ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
84202025-06-03T00:59:40.847ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
84212025-06-03T00:59:40.849ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: ec5e33f9-c697-46e2-a93e-a745336bf9dc, rack_id: d6831ba0-55f0-4191-8dc2-ea15e55ab0ae, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(9378a284-0ee9-48e2-80a0-3bcaadb1c4e2) } rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae switch_location = Switch0
84222025-06-03T00:59:40.849ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
84232025-06-03T00:59:40.849ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
84242025-06-03T00:59:40.857ZDEBGnexus (BackgroundTasks): client response SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "9ebcc846-88da-4a6d-abff-b26e44b289fd", "content-length": "372", "date": "Tue, 03 Jun 2025 00:59:40 GMT"} })
84252025-06-03T00:59:40.858ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
84262025-06-03T00:59:40.858ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
84272025-06-03T00:59:40.863ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1832 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
84282025-06-03T00:59:40.863ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1859 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 9378a284-0ee9-48e2-80a0-3bcaadb1c4e2
84292025-06-03T00:59:40.863ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
84302025-06-03T00:59:40.863ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
84312025-06-03T00:59:40.863ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
84322025-06-03T00:59:40.863ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
84332025-06-03T00:59:40.863ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
84342025-06-03T00:59:40.863ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2006 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
84352025-06-03T00:59:40.863ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:408 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
84362025-06-03T00:59:40.863ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2042 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
84372025-06-03T00:59:40.864ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae
84382025-06-03T00:59:40.864ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
84392025-06-03T00:59:40.864ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
84402025-06-03T00:59:40.867ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae switch_location = Switch0
84412025-06-03T00:59:40.867ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
84422025-06-03T00:59:40.879ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
84432025-06-03T00:59:40.879ZDEBGnexus (BackgroundTasks): client request SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
84442025-06-03T00:59:40.879ZDEBGnexus (BackgroundTasks): client response SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "255886b3-6de1-442e-ad09-95596a6a0616", "date": "Tue, 03 Jun 2025 00:59:40 GMT"} })
84452025-06-03T00:59:40.879ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
84462025-06-03T00:59:40.879ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
84472025-06-03T00:59:40.881ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
84482025-06-03T00:59:40.881ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
84492025-06-03T00:59:40.883ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
84502025-06-03T00:59:40.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
84512025-06-03T00:59:40.884ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
84522025-06-03T00:59:40.885ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
84532025-06-03T00:59:40.885ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
84542025-06-03T00:59:40.886ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
84552025-06-03T00:59:40.886ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
84562025-06-03T00:59:40.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
84572025-06-03T00:59:40.888ZINFOnexus (BackgroundTasks): VPC route manager sled afe89ed7-19b7-479b-81eb-05c988ebc66d push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
84582025-06-03T00:59:40.888ZDEBGnexus (BackgroundTasks): client request SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
84592025-06-03T00:59:40.890ZDEBGnexus (BackgroundTasks): client response SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "88285894-72c1-492f-a7da-a9068e7f601c", "date": "Tue, 03 Jun 2025 00:59:40 GMT"} })
84602025-06-03T00:59:40.892ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 389.879596ms iteration = 18 status = {}
84612025-06-03T00:59:40.921ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
84622025-06-03T00:59:40.923ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
84632025-06-03T00:59:40.923ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
84642025-06-03T00:59:40.923ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
84652025-06-03T00:59:40.955ZDEBGnexus (BackgroundTasks): client request SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
84662025-06-03T00:59:40.956ZDEBGnexus (BackgroundTasks): client response SledAgent = afe89ed7-19b7-479b-81eb-05c988ebc66d background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "bb08f3de-3318-4877-9a4e-4a1c06649cf9", "content-length": "691", "date": "Tue, 03 Jun 2025 00:59:40 GMT"} })
84672025-06-03T00:59:40.956ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
84682025-06-03T00:59:40.956ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
84692025-06-03T00:59:40.956ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
84702025-06-03T00:59:40.957ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 455.291448ms iteration = 18 status = {}
84712025-06-03T00:59:41.046ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "1150cefa-235b-476e-ac1a-b1c798558c8d", "date": "Tue, 03 Jun 2025 00:59:40 GMT"} })
84722025-06-03T00:59:41.046ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae switch = Switch0
84732025-06-03T00:59:41.046ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
84742025-06-03T00:59:41.047ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = d6831ba0-55f0-4191-8dc2-ea15e55ab0ae result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "537a5f0d-e3d3-4175-bf07-a08f82b59dee", "content-length": "682", "date": "Tue, 03 Jun 2025 00:59:40 GMT"} })
84752025-06-03T00:59:41.047ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
84762025-06-03T00:59:41.047ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
84772025-06-03T00:59:41.051ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
84782025-06-03T00:59:41.051ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
84792025-06-03T00:59:41.054ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
84802025-06-03T00:59:41.055ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
84812025-06-03T00:59:41.057ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
84822025-06-03T00:59:41.057ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
84832025-06-03T00:59:41.064ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 561.903828ms iteration = 18 status = {}
84842025-06-03T00:59:41.228ZDEBGnexus (producer-registration-task): registering as oximeter producer with Nexus registration_count = 3
84852025-06-03T00:59:41.228ZDEBGnexus (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::a]:12221
84862025-06-03T00:59:41.259ZDEBGnexus (producer-registration-task): client request body = Some(Body) method = POST uri = http://[fd00:1122:3344:101::a]:12221/metrics/producers
84872025-06-03T00:59:41.259ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:1025 local_addr = [fd00:1122:3344:101::a]:12221 remote_addr = [fd00:1122:3344:101::a]:55101
84882025-06-03T00:59:41.259ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = POST remote_addr = [fd00:1122:3344:101::a]:55101 req_id = 22240e8d-4ee7-454a-8739-2aa98cb71581 roles = RoleSet { roles: {} } uri = /metrics/producers
84892025-06-03T00:59:41.259ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = POST remote_addr = [fd00:1122:3344:101::a]:55101 req_id = 22240e8d-4ee7-454a-8739-2aa98cb71581 resource = Database result = Ok(()) uri = /metrics/producers
84902025-06-03T00:59:41.298ZINFOnexus: assigned collector to new producer collector_id = 292a19e1-4305-4e49-b11e-08ac74714bf3 file = nexus/src/app/oximeter.rs:76 producer_id = a2861392-5fd2-462b-84e5-6f36c716263a
84912025-06-03T00:59:41.298ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 39532 local_addr = [fd00:1122:3344:101::a]:12221 method = POST remote_addr = [fd00:1122:3344:101::a]:55101 req_id = 22240e8d-4ee7-454a-8739-2aa98cb71581 response_code = 201 uri = /metrics/producers
84922025-06-03T00:59:41.298ZDEBGnexus (producer-registration-task): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::a]:12221/metrics/producers", status: 201, headers: {"content-type": "application/json", "x-request-id": "22240e8d-4ee7-454a-8739-2aa98cb71581", "content-length": "41", "date": "Tue, 03 Jun 2025 00:59:41 GMT"} })
84932025-06-03T00:59:41.299ZDEBGnexus (producer-registration-task): successfully registered with Nexus, pausing until time to renew lease lease_duration = 600s wait_period = 150s
84942025-06-03T00:59:45.670ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 303 local_addr = [fd00:1122:3344:101::a]:63576 method = GET remote_addr = [fd00:1122:3344:101::d]:61315 req_id = 7b5f5ae9-1ab9-49ab-96a6-a12b88706346 response_code = 200 uri = /a2861392-5fd2-462b-84e5-6f36c716263a
84952025-06-03T00:59:50.502ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 53 reason = Timeout
84962025-06-03T00:59:50.503ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
84972025-06-03T00:59:50.503ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 53 reason = Timeout
84982025-06-03T00:59:50.503ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
84992025-06-03T00:59:50.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
85002025-06-03T00:59:50.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
85012025-06-03T00:59:50.539ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
85022025-06-03T00:59:50.539ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
85032025-06-03T00:59:50.541ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
85042025-06-03T00:59:50.543ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
85052025-06-03T00:59:50.543ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
85062025-06-03T00:59:50.544ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
85072025-06-03T00:59:50.546ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
85082025-06-03T00:59:50.546ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
85092025-06-03T00:59:50.548ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
85102025-06-03T00:59:50.551ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 4ec93699-353f-4b9e-ae06-25355fdb069e, lookup_type: ById(4ec93699-353f-4b9e-ae06-25355fdb069e) } result = Ok(())
85112025-06-03T00:59:50.551ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
85122025-06-03T00:59:50.551ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
85132025-06-03T00:59:50.563ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 4ec93699-353f-4b9e-ae06-25355fdb069e original_time_created = 2025-06-03 00:51:07.059326 UTC target_id = 4ec93699-353f-4b9e-ae06-25355fdb069e time_created = 2025-06-03 00:51:07.059326 UTC
85142025-06-03T00:59:50.563ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 60.918962ms iteration = 53 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"4ec93699-353f-4b9e-ae06-25355fdb069e","time_created":"2025-06-03T00:51:07.059326Z"}
85152025-06-03T00:59:50.577ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 74.316357ms iteration = 53 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
85162025-06-03T00:59:55.659ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:1025 local_addr = [fd00:1122:3344:101::a]:12221 remote_addr = [fd00:1122:3344:101::d]:42933
85172025-06-03T00:59:55.659ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42933 req_id = 88fee18c-4a82-46a3-81ea-2c49f630ccd0 roles = RoleSet { roles: {} } uri = /metrics/collectors/292a19e1-4305-4e49-b11e-08ac74714bf3/producers?sort_by=id_ascending
85182025-06-03T00:59:55.659ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42933 req_id = 88fee18c-4a82-46a3-81ea-2c49f630ccd0 resource = Database result = Ok(()) uri = /metrics/collectors/292a19e1-4305-4e49-b11e-08ac74714bf3/producers?sort_by=id_ascending
85192025-06-03T00:59:55.662ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 3076 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42933 req_id = 88fee18c-4a82-46a3-81ea-2c49f630ccd0 response_code = 200 uri = /metrics/collectors/292a19e1-4305-4e49-b11e-08ac74714bf3/producers?sort_by=id_ascending
85202025-06-03T00:59:55.662ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42933 req_id = c353d02d-d028-4263-bb03-80087496d336 roles = RoleSet { roles: {} } uri = /metrics/collectors/292a19e1-4305-4e49-b11e-08ac74714bf3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmZTEyZmZjMS1mMmFjLTRiZjMtOTY3Mi1mYzE4NWY4MmM1OWIifX0%3D
85212025-06-03T00:59:55.662ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42933 req_id = c353d02d-d028-4263-bb03-80087496d336 resource = Database result = Ok(()) uri = /metrics/collectors/292a19e1-4305-4e49-b11e-08ac74714bf3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmZTEyZmZjMS1mMmFjLTRiZjMtOTY3Mi1mYzE4NWY4MmM1OWIifX0%3D
85222025-06-03T00:59:55.664ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 2066 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42933 req_id = c353d02d-d028-4263-bb03-80087496d336 response_code = 200 uri = /metrics/collectors/292a19e1-4305-4e49-b11e-08ac74714bf3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmZTEyZmZjMS1mMmFjLTRiZjMtOTY3Mi1mYzE4NWY4MmM1OWIifX0%3D
85232025-06-03T00:59:55.670ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.2/src/server.rs:867 latency_us = 320 local_addr = [fd00:1122:3344:101::a]:63576 method = GET remote_addr = [fd00:1122:3344:101::d]:61315 req_id = bd137bea-7d93-43bf-a374-1348e369fb98 response_code = 200 uri = /a2861392-5fd2-462b-84e5-6f36c716263a
85242025-06-03T01:00:00.502ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 54 reason = Timeout
85252025-06-03T01:00:00.502ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
85262025-06-03T01:00:00.502ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 54 reason = Timeout
85272025-06-03T01:00:00.502ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
85282025-06-03T01:00:00.502ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
85292025-06-03T01:00:00.502ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
85302025-06-03T01:00:00.505ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
85312025-06-03T01:00:00.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
85322025-06-03T01:00:00.505ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
85332025-06-03T01:00:00.508ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
85342025-06-03T01:00:00.508ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
85352025-06-03T01:00:00.509ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
85362025-06-03T01:00:00.511ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
85372025-06-03T01:00:00.512ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
85382025-06-03T01:00:00.514ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
85392025-06-03T01:00:00.518ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 4ec93699-353f-4b9e-ae06-25355fdb069e, lookup_type: ById(4ec93699-353f-4b9e-ae06-25355fdb069e) } result = Ok(())
85402025-06-03T01:00:00.518ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
85412025-06-03T01:00:00.518ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
85422025-06-03T01:00:00.532ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 4ec93699-353f-4b9e-ae06-25355fdb069e original_time_created = 2025-06-03 00:51:07.059326 UTC target_id = 4ec93699-353f-4b9e-ae06-25355fdb069e time_created = 2025-06-03 00:51:07.059326 UTC
85432025-06-03T01:00:00.532ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 30.731424ms iteration = 54 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"4ec93699-353f-4b9e-ae06-25355fdb069e","time_created":"2025-06-03T00:51:07.059326Z"}
85442025-06-03T01:00:00.552ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 50.786487ms iteration = 54 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}