Max level shown:
1 [ Aug 26 21:45:41 Enabled. ]
2 [ Aug 26 21:45:41 Rereading configuration. ]
3 [ Aug 26 21:45:46 Rereading configuration. ]
4 [ Aug 26 21:45:57 Executing start method ("ctrun -l child -o noorphan,regent /opt/oxide/omicron-nexus/bin/nexus /var/svc/manifest/site/nexus/config.toml &"). ]
5 [ Aug 26 21:45:57 Method "start" exited with status 0. ]
6 note: configured to log to "/dev/stdout"
72025-08-26T21:45:58.545ZDEBGnexus: registered DTrace probes
82025-08-26T21:45:58.546ZINFOnexus: setting up nexus server file = nexus/src/lib.rs:79
92025-08-26T21:45:58.559ZINFOnexus (ServerContext): registering Oso class class = Action file = nexus/auth/src/authz/oso_generic.rs:68
102025-08-26T21:45:58.559ZINFOnexus (ServerContext): registering Oso class class = AuditLog file = nexus/auth/src/authz/oso_generic.rs:68
112025-08-26T21:45:58.559ZINFOnexus (ServerContext): registering Oso class class = AnyActor file = nexus/auth/src/authz/oso_generic.rs:68
122025-08-26T21:45:58.560ZINFOnexus (ServerContext): registering Oso class class = AuthenticatedActor file = nexus/auth/src/authz/oso_generic.rs:68
132025-08-26T21:45:58.560ZINFOnexus (ServerContext): registering Oso class class = BlueprintConfig file = nexus/auth/src/authz/oso_generic.rs:68
142025-08-26T21:45:58.560ZINFOnexus (ServerContext): registering Oso class class = Database file = nexus/auth/src/authz/oso_generic.rs:68
152025-08-26T21:45:58.560ZINFOnexus (ServerContext): registering Oso class class = DnsConfig file = nexus/auth/src/authz/oso_generic.rs:68
162025-08-26T21:45:58.560ZINFOnexus (ServerContext): registering Oso class class = Fleet file = nexus/auth/src/authz/oso_generic.rs:68
172025-08-26T21:45:58.560ZINFOnexus (ServerContext): registering Oso class class = Inventory file = nexus/auth/src/authz/oso_generic.rs:68
182025-08-26T21:45:58.560ZINFOnexus (ServerContext): registering Oso class class = IpPoolList file = nexus/auth/src/authz/oso_generic.rs:68
192025-08-26T21:45:58.560ZINFOnexus (ServerContext): registering Oso class class = ConsoleSessionList file = nexus/auth/src/authz/oso_generic.rs:68
202025-08-26T21:45:58.560ZINFOnexus (ServerContext): registering Oso class class = DeviceAuthRequestList file = nexus/auth/src/authz/oso_generic.rs:68
212025-08-26T21:45:58.560ZINFOnexus (ServerContext): registering Oso class class = QuiesceState file = nexus/auth/src/authz/oso_generic.rs:68
222025-08-26T21:45:58.560ZINFOnexus (ServerContext): registering Oso class class = SiloCertificateList file = nexus/auth/src/authz/oso_generic.rs:68
232025-08-26T21:45:58.560ZINFOnexus (ServerContext): registering Oso class class = SiloIdentityProviderList file = nexus/auth/src/authz/oso_generic.rs:68
242025-08-26T21:45:58.560ZINFOnexus (ServerContext): registering Oso class class = SiloUserList file = nexus/auth/src/authz/oso_generic.rs:68
252025-08-26T21:45:58.560ZINFOnexus (ServerContext): registering Oso class class = SiloUserSessionList file = nexus/auth/src/authz/oso_generic.rs:68
262025-08-26T21:45:58.560ZINFOnexus (ServerContext): registering Oso class class = SiloUserTokenList file = nexus/auth/src/authz/oso_generic.rs:68
272025-08-26T21:45:58.560ZINFOnexus (ServerContext): registering Oso class class = UpdateTrustRootList file = nexus/auth/src/authz/oso_generic.rs:68
282025-08-26T21:45:58.560ZINFOnexus (ServerContext): registering Oso class class = TargetReleaseConfig file = nexus/auth/src/authz/oso_generic.rs:68
292025-08-26T21:45:58.560ZINFOnexus (ServerContext): registering Oso class class = AlertClassList file = nexus/auth/src/authz/oso_generic.rs:68
302025-08-26T21:45:58.560ZINFOnexus (ServerContext): registering Oso class class = Project file = nexus/auth/src/authz/oso_generic.rs:68
312025-08-26T21:45:58.560ZINFOnexus (ServerContext): registering Oso class class = Disk file = nexus/auth/src/authz/oso_generic.rs:68
322025-08-26T21:45:58.560ZINFOnexus (ServerContext): registering Oso class class = Snapshot file = nexus/auth/src/authz/oso_generic.rs:68
332025-08-26T21:45:58.560ZINFOnexus (ServerContext): registering Oso class class = ProjectImage file = nexus/auth/src/authz/oso_generic.rs:68
342025-08-26T21:45:58.560ZINFOnexus (ServerContext): registering Oso class class = AffinityGroup file = nexus/auth/src/authz/oso_generic.rs:68
352025-08-26T21:45:58.560ZINFOnexus (ServerContext): registering Oso class class = AntiAffinityGroup file = nexus/auth/src/authz/oso_generic.rs:68
362025-08-26T21:45:58.560ZINFOnexus (ServerContext): registering Oso class class = Instance file = nexus/auth/src/authz/oso_generic.rs:68
372025-08-26T21:45:58.560ZINFOnexus (ServerContext): registering Oso class class = IpPool file = nexus/auth/src/authz/oso_generic.rs:68
382025-08-26T21:45:58.560ZINFOnexus (ServerContext): registering Oso class class = InstanceNetworkInterface file = nexus/auth/src/authz/oso_generic.rs:68
392025-08-26T21:45:58.560ZINFOnexus (ServerContext): registering Oso class class = Vpc file = nexus/auth/src/authz/oso_generic.rs:68
402025-08-26T21:45:58.560ZINFOnexus (ServerContext): registering Oso class class = VpcRouter file = nexus/auth/src/authz/oso_generic.rs:68
412025-08-26T21:45:58.560ZINFOnexus (ServerContext): registering Oso class class = InternetGateway file = nexus/auth/src/authz/oso_generic.rs:68
422025-08-26T21:45:58.560ZINFOnexus (ServerContext): registering Oso class class = InternetGatewayIpPool file = nexus/auth/src/authz/oso_generic.rs:68
432025-08-26T21:45:58.560ZINFOnexus (ServerContext): registering Oso class class = InternetGatewayIpAddress file = nexus/auth/src/authz/oso_generic.rs:68
442025-08-26T21:45:58.560ZINFOnexus (ServerContext): registering Oso class class = RouterRoute file = nexus/auth/src/authz/oso_generic.rs:68
452025-08-26T21:45:58.560ZINFOnexus (ServerContext): registering Oso class class = VpcSubnet file = nexus/auth/src/authz/oso_generic.rs:68
462025-08-26T21:45:58.561ZINFOnexus (ServerContext): registering Oso class class = FloatingIp file = nexus/auth/src/authz/oso_generic.rs:68
472025-08-26T21:45:58.561ZINFOnexus (ServerContext): registering Oso class class = Image file = nexus/auth/src/authz/oso_generic.rs:68
482025-08-26T21:45:58.561ZINFOnexus (ServerContext): registering Oso class class = SiloImage file = nexus/auth/src/authz/oso_generic.rs:68
492025-08-26T21:45:58.561ZINFOnexus (ServerContext): registering Oso class class = AddressLot file = nexus/auth/src/authz/oso_generic.rs:68
502025-08-26T21:45:58.561ZINFOnexus (ServerContext): registering Oso class class = Blueprint file = nexus/auth/src/authz/oso_generic.rs:68
512025-08-26T21:45:58.561ZINFOnexus (ServerContext): registering Oso class class = LoopbackAddress file = nexus/auth/src/authz/oso_generic.rs:68
522025-08-26T21:45:58.561ZINFOnexus (ServerContext): registering Oso class class = Certificate file = nexus/auth/src/authz/oso_generic.rs:68
532025-08-26T21:45:58.561ZINFOnexus (ServerContext): registering Oso class class = ConsoleSession file = nexus/auth/src/authz/oso_generic.rs:68
542025-08-26T21:45:58.561ZINFOnexus (ServerContext): registering Oso class class = DeviceAuthRequest file = nexus/auth/src/authz/oso_generic.rs:68
552025-08-26T21:45:58.561ZINFOnexus (ServerContext): registering Oso class class = DeviceAccessToken file = nexus/auth/src/authz/oso_generic.rs:68
562025-08-26T21:45:58.561ZINFOnexus (ServerContext): registering Oso class class = PhysicalDisk file = nexus/auth/src/authz/oso_generic.rs:68
572025-08-26T21:45:58.561ZINFOnexus (ServerContext): registering Oso class class = Rack file = nexus/auth/src/authz/oso_generic.rs:68
582025-08-26T21:45:58.561ZINFOnexus (ServerContext): registering Oso class class = SshKey file = nexus/auth/src/authz/oso_generic.rs:68
592025-08-26T21:45:58.561ZINFOnexus (ServerContext): registering Oso class class = Silo file = nexus/auth/src/authz/oso_generic.rs:68
602025-08-26T21:45:58.561ZINFOnexus (ServerContext): registering Oso class class = SiloUser file = nexus/auth/src/authz/oso_generic.rs:68
612025-08-26T21:45:58.561ZINFOnexus (ServerContext): registering Oso class class = SiloGroup file = nexus/auth/src/authz/oso_generic.rs:68
622025-08-26T21:45:58.561ZINFOnexus (ServerContext): registering Oso class class = SupportBundle file = nexus/auth/src/authz/oso_generic.rs:68
632025-08-26T21:45:58.561ZINFOnexus (ServerContext): registering Oso class class = IdentityProvider file = nexus/auth/src/authz/oso_generic.rs:68
642025-08-26T21:45:58.561ZINFOnexus (ServerContext): registering Oso class class = SamlIdentityProvider file = nexus/auth/src/authz/oso_generic.rs:68
652025-08-26T21:45:58.561ZINFOnexus (ServerContext): registering Oso class class = Sled file = nexus/auth/src/authz/oso_generic.rs:68
662025-08-26T21:45:58.561ZINFOnexus (ServerContext): registering Oso class class = TufRepo file = nexus/auth/src/authz/oso_generic.rs:68
672025-08-26T21:45:58.561ZINFOnexus (ServerContext): registering Oso class class = TufArtifact file = nexus/auth/src/authz/oso_generic.rs:68
682025-08-26T21:45:58.561ZINFOnexus (ServerContext): registering Oso class class = TufTrustRoot file = nexus/auth/src/authz/oso_generic.rs:68
692025-08-26T21:45:58.561ZINFOnexus (ServerContext): registering Oso class class = Alert file = nexus/auth/src/authz/oso_generic.rs:68
702025-08-26T21:45:58.561ZINFOnexus (ServerContext): registering Oso class class = AlertReceiver file = nexus/auth/src/authz/oso_generic.rs:68
712025-08-26T21:45:58.561ZINFOnexus (ServerContext): registering Oso class class = WebhookSecret file = nexus/auth/src/authz/oso_generic.rs:68
722025-08-26T21:45:58.561ZINFOnexus (ServerContext): registering Oso class class = Zpool file = nexus/auth/src/authz/oso_generic.rs:68
732025-08-26T21:45:58.561ZINFOnexus (ServerContext): registering Oso class class = Service file = nexus/auth/src/authz/oso_generic.rs:68
742025-08-26T21:45:58.561ZINFOnexus (ServerContext): registering Oso class class = UserBuiltin file = nexus/auth/src/authz/oso_generic.rs:68
752025-08-26T21:45:58.561ZINFOnexus (ServerContext): full Oso configuration config = #\n# Oso configuration for Omicron\n# This file is augmented by generated snippets.\n#\n\n#\n# ACTOR TYPES AND BASIC RULES\n#\n\n# `AnyActor` includes both authenticated and unauthenticated users.\nactor AnyActor {}\n\n# An `AuthenticatedActor` has an identity in the system. All of our operations\n# today require that an actor be authenticated.\nactor AuthenticatedActor {}\n\n# For any resource, `actor` can perform action `action` on it if they're\n# authenticated and their role(s) give them the corresponding permission on that\n# resource.\nallow(actor: AnyActor, action: Action, resource) if\n actor.authenticated and\n has_permission(actor.authn_actor.unwrap(), action.to_perm(), resource);\n\n# Define role relationships\nhas_role(actor: AuthenticatedActor, role: String, resource: Resource)\n\tif resource.has_role(actor, role);\n\n#\n# ROLES AND PERMISSIONS IN THE FLEET/SILO/PROJECT HIERARCHY\n#\n# We define the following permissions for most resources in the system:\n#\n# - "create_child": required to create child resources (of any type)\n#\n# - "list_children": required to list child resources (of all types) of a\n# resource\n#\n# - "modify": required to modify or delete a resource\n#\n# - "read": required to read a resource\n#\n# We define the following predefined roles for only a few high-level resources:\n# the Fleet (see below), Silo, Organization, and Project. The specific roles\n# are oriented around intended use-cases:\n#\n# - "admin": has all permissions on the resource\n#\n# - "collaborator": has "read", "list_children", and "create_child", plus\n# the "admin" role for child resources. The idea is that if you're an\n# Organization Collaborator, you have full control over the Projects within\n# the Organization, but you cannot modify or delete the Organization itself.\n#\n# - "viewer": has "read" and "list_children" on a resource\n#\n# Below the Project level, permissions are granted via roles at the Project\n# level. For example, for someone to be able to create, modify, or delete any\n# Instances, they must be granted project.collaborator, which means they can\n# create, modify, or delete _all_ resources in the Project.\n#\n# The complete set of predefined roles:\n#\n# - fleet.admin (superuser for the whole system)\n# - fleet.collaborator (can manage Silos)\n# - fleet.viewer (can read most non-siloed resources in the system)\n# - silo.admin (superuser for the silo)\n# - silo.collaborator (can create and own Organizations)\n# - silo.viewer (can read most resources within the Silo)\n# - organization.admin (complete control over an organization)\n# - organization.collaborator (can manage Projects)\n# - organization.viewer (can read most resources within the Organization)\n# - project.admin (complete control over a Project)\n# - project.collaborator (can manage all resources within the Project)\n# - project.viewer (can read most resources within the Project)\n#\n# Outside the Silo/Organization/Project hierarchy, we (currently) treat most\n# resources as nested under Fleet or else a synthetic resource (see below). We\n# do not yet support role assignments on anything other than Fleet, Silo,\n# Organization, or Project.\n#\n\n# "Fleet" is a global singleton representing the whole system. The name comes\n# from the idea described in RFD 24, but it's not quite right. This probably\n# should be more like "Region" or "AvailabilityZone". The precise boundaries\n# have not yet been figured out.\nresource Fleet {\n\tpermissions = [\n\t "list_children",\n\t "modify",\n\t "read",\n\t "create_child",\n\t];\n\n\troles = [\n\t # Roles that can be attached by users\n\t "admin",\n\t "collaborator",\n\t "viewer",\n\n\t # Internal-only roles\n\t "external-authenticator"\n\t];\n\n\t# Roles implied by other roles on this resource\n\t"viewer" if "collaborator";\n\t"collaborator" if "admin";\n\n\t# Permissions granted directly by roles on this resource\n\t"list_children" if "viewer";\n\t"read" if "viewer";\n\t"create_child" if "collaborator";\n\t"modify" if "admin";\n}\n\n# For fleets specifically, roles can be conferred by roles on the user's Silo.\nhas_role(actor: AuthenticatedActor, role: String, _: Fleet) if\n\tsilo_role in actor.confers_fleet_role(role) and\n\thas_role(actor, silo_role, actor.silo.unwrap());\n\nresource Silo {\n\tpermissions = [\n\t "list_children",\n\t "modify",\n\t "read",\n\t "create_child",\n\t];\n\troles = [ "admin", "collaborator", "viewer" ];\n\n\t# Roles implied by other roles on this resource\n\t"viewer" if "collaborator";\n\t"collaborator" if "admin";\n\n\t# Permissions granted directly by roles on this resource\n\t"list_children" if "viewer";\n\t"read" if "viewer";\n\n\t"create_child" if "collaborator";\n\t"modify" if "admin";\n\n\t# Permissions implied by roles on this resource's parent (Fleet). Fleet\n\t# privileges allow a user to see and potentially administer the Silo,\n\t# but they do not give anyone permission to look at anything inside the\n\t# Silo. To achieve this, we use permission rules here. (If we granted\n\t# Fleet administrators _roles_ on the Silo, then those would cascade\n\t# into the Silo as well.)\n\trelations = { parent_fleet: Fleet };\n\t"read" if "viewer" on "parent_fleet";\n\t"modify" if "collaborator" on "parent_fleet";\n\n\t# external authenticator has to create silo users\n\t"list_children" if "external-authenticator" on "parent_fleet";\n\t"create_child" if "external-authenticator" on "parent_fleet";\n}\n\nhas_relation(fleet: Fleet, "parent_fleet", silo: Silo)\n\tif silo.fleet = fleet;\n\n# As a special case, all authenticated users can read their own Silo. That's\n# not quite the same as having the "viewer" role. For example, they cannot list\n# Organizations in the Silo.\n#\n# One reason this is necessary is because if an unprivileged user tries to\n# create an Organization using "POST /organizations", they should get back a 403\n# (which implies they're able to see /organizations, which is essentially seeing\n# the Silo itself) rather than a 404. This behavior isn't a hard constraint\n# (i.e., you could reasonably get a 404 for an API you're not allowed to call).\n# Nor is the implementation (i.e., we could special-case this endpoint somehow).\n# But granting this permission is the simplest way to keep this endpoint's\n# behavior consistent with the rest of the API.\n#\n# This rule is also used to determine if a user can list the identity providers\n# in the Silo (which they should be able to), since that's predicated on being\n# able to read the Silo.\n#\n# It's unclear what else would break if users couldn't see their own Silo.\nhas_permission(actor: AuthenticatedActor, "read", silo: Silo)\n\tif silo in actor.silo;\n\nresource Project {\n\tpermissions = [\n\t "list_children",\n\t "modify",\n\t "read",\n\t "create_child",\n\t];\n\troles = [ "admin", "collaborator", "viewer" ];\n\n\t# Roles implied by other roles on this resource\n\t"viewer" if "collaborator";\n\t"collaborator" if "admin";\n\n\t# Permissions granted directly by roles on this resource\n\t"list_children" if "viewer";\n\t"read" if "viewer";\n\t"create_child" if "collaborator";\n\t"modify" if "admin";\n\n\t# Roles implied by roles on this resource's parent (Silo)\n\trelations = { parent_silo: Silo };\n\t"admin" if "collaborator" on "parent_silo";\n\t"viewer" if "viewer" on "parent_silo";\n}\nhas_relation(silo: Silo, "parent_silo", project: Project)\n\tif project.silo = silo;\n\n#\n# GENERAL RESOURCES OUTSIDE THE SILO/PROJECT HIERARCHY\n#\n# Many resources use snippets of Polar generated by the `authz_resource!` Rust\n# macro. Some resources require custom Polar code. Those appear here.\n#\n\nresource Certificate {\n\tpermissions = [ "read", "modify" ];\n\trelations = { parent_silo: Silo, parent_fleet: Fleet };\n\n\t# Fleet-level and silo-level roles both grant privileges on certificates.\n\t"read" if "admin" on "parent_silo";\n\t"modify" if "admin" on "parent_silo";\n\t"read" if "admin" on "parent_fleet";\n\t"modify" if "admin" on "parent_fleet";\n}\nhas_relation(silo: Silo, "parent_silo", certificate: Certificate)\n\tif certificate.silo = silo;\nhas_relation(fleet: Fleet, "parent_fleet", certificate: Certificate)\n\tif certificate.silo.fleet = fleet;\n\nresource SiloUser {\n\tpermissions = [\n\t "list_children",\n\t "modify",\n\t "read",\n\t "create_child",\n\t];\n\n\t# Fleet and Silo administrators can manage a Silo's users. This is one\n\t# of the only areas of Silo configuration that Fleet Administrators have\n\t# permissions on.\n\trelations = { parent_silo: Silo, parent_fleet: Fleet };\n\t"list_children" if "read" on "parent_silo";\n\t"read" if "read" on "parent_silo";\n\t"modify" if "admin" on "parent_silo";\n\t"create_child" if "admin" on "parent_silo";\n\t"list_children" if "read" on "parent_fleet";\n\t"read" if "read" on "parent_fleet";\n\t"modify" if "admin" on "parent_fleet";\n\t"create_child" if "admin" on "parent_fleet";\n}\nhas_relation(silo: Silo, "parent_silo", user: SiloUser)\n\tif user.silo = silo;\nhas_relation(fleet: Fleet, "parent_fleet", user: SiloUser)\n\tif user.silo.fleet = fleet;\n\n# authenticated actors have all permissions on themselves\nhas_permission(actor: AuthenticatedActor, _perm: String, silo_user: SiloUser)\n if actor.equals_silo_user(silo_user);\n\nhas_permission(actor: AuthenticatedActor, "read", silo_user: SiloUser)\n if silo_user.silo in actor.silo;\n\nresource SiloGroup {\n\tpermissions = [\n\t "list_children",\n\t "modify",\n\t "read",\n\t "create_child",\n\t];\n\n\trelations = { parent_silo: Silo };\n\t"list_children" if "read" on "parent_silo";\n\t"read" if "read" on "parent_silo";\n\t"modify" if "admin" on "parent_silo";\n\t"create_child" if "admin" on "parent_silo";\n}\nhas_relation(silo: Silo, "parent_silo", group: SiloGroup)\n\tif group.silo = silo;\n\nresource SshKey {\n\tpermissions = [ "read", "modify" ];\n\trelations = { silo_user: SiloUser };\n\n\t"read" if "read" on "silo_user";\n\t"modify" if "modify" on "silo_user";\n}\nhas_relation(user: SiloUser, "silo_user", ssh_key: SshKey)\n\tif ssh_key.silo_user = user;\n\nresource IdentityProvider {\n\tpermissions = [\n\t "read",\n\t "modify",\n\t "create_child",\n\t "list_children",\n\t];\n\trelations = { parent_silo: Silo, parent_fleet: Fleet };\n\n\t# Silo-level roles grant privileges on identity providers.\n\t"read" if "viewer" on "parent_silo";\n\t"list_children" if "viewer" on "parent_silo";\n\t"modify" if "admin" on "parent_silo";\n\t"create_child" if "admin" on "parent_silo";\n\n\t# Fleet-level roles also grant privileges on identity providers.\n\t"read" if "viewer" on "parent_fleet";\n\t"list_children" if "viewer" on "parent_fleet";\n\t"modify" if "admin" on "parent_fleet";\n\t"create_child" if "admin" on "parent_fleet";\n}\nhas_relation(silo: Silo, "parent_silo", identity_provider: IdentityProvider)\n\tif identity_provider.silo = silo;\nhas_relation(fleet: Fleet, "parent_fleet", collection: IdentityProvider)\n\tif collection.silo.fleet = fleet;\n\nresource SamlIdentityProvider {\n\tpermissions = [\n\t "read",\n\t "modify",\n\t "create_child",\n\t "list_children",\n\t];\n\trelations = { parent_silo: Silo, parent_fleet: Fleet };\n\n\t# Silo-level roles grant privileges on identity providers.\n\t"read" if "viewer" on "parent_silo";\n\t"list_children" if "viewer" on "parent_silo";\n\t"modify" if "admin" on "parent_silo";\n\t"create_child" if "admin" on "parent_silo";\n\n\t# Fleet-level roles also grant privileges on identity providers.\n\t"read" if "viewer" on "parent_fleet";\n\t"list_children" if "viewer" on "parent_fleet";\n\t"modify" if "admin" on "parent_fleet";\n\t"create_child" if "admin" on "parent_fleet";\n}\nhas_relation(silo: Silo, "parent_silo", saml_identity_provider: SamlIdentityProvider)\n\tif saml_identity_provider.silo = silo;\nhas_relation(fleet: Fleet, "parent_fleet", collection: SamlIdentityProvider)\n\tif collection.silo.fleet = fleet;\n\n#\n# SYNTHETIC RESOURCES OUTSIDE THE SILO HIERARCHY\n#\n# The resources here do not correspond to anything that appears explicitly in\n# the API or is stored in the database. These are used either at the top level\n# of the API path (e.g., "/v1/system/ip-pools") or as an implementation detail of the system\n# (in the case of console sessions and "Database"). The policies are\n# either statically-defined in this file or driven by role assignments on the\n# Fleet. None of these resources defines their own roles.\n#\n\n# Describes the quiesce state of a particular Nexus instance.\n#\n# These authz checks must not require the database. We grant this directly to\n# callers of the internal API.\nresource QuiesceState {\n\tpermissions = [ "read", "modify" ];\n}\nhas_permission(USER_INTERNAL_API: AuthenticatedActor, "read", _q: QuiesceState);\nhas_permission(\n USER_INTERNAL_API: AuthenticatedActor,\n "modify",\n _q: QuiesceState\n);\n\n# Describes the policy for reading and modifying DNS configuration\n# (both internal and external)\nresource DnsConfig {\n\tpermissions = [ "read", "modify" ];\n\trelations = { parent_fleet: Fleet };\n\t# "external-authenticator" requires these permissions because that's the\n\t# context that Nexus uses when creating and deleting Silos. These\n\t# operations necessarily need to read and modify DNS configuration.\n\t"read" if "external-authenticator" on "parent_fleet";\n\t"modify" if "external-authenticator" on "parent_fleet";\n\t# "admin" on the parent fleet also gets these permissions, primarily for\n\t# the test suite.\n\t"read" if "admin" on "parent_fleet";\n\t"modify" if "admin" on "parent_fleet";\n}\nhas_relation(fleet: Fleet, "parent_fleet", dns_config: DnsConfig)\n\tif dns_config.fleet = fleet;\n\n# Describes the policy for accessing blueprints\nresource BlueprintConfig {\n\tpermissions = [\n\t "list_children", # list blueprints\n\t "create_child", # create blueprint\n\t "read", # read the current target\n\t "modify", # change the current target\n\t];\n\n\trelations = { parent_fleet: Fleet };\n\t"create_child" if "admin" on "parent_fleet";\n\t"modify" if "admin" on "parent_fleet";\n\t"list_children" if "viewer" on "parent_fleet";\n\t"read" if "viewer" on "parent_fleet";\n}\nhas_relation(fleet: Fleet, "parent_fleet", list: BlueprintConfig)\n\tif list.fleet = fleet;\n\n# Describes the policy for accessing "/v1/system/update/trust-roots" in the API\nresource UpdateTrustRootList {\n\tpermissions = [ "list_children", "create_child" ];\n\trelations = { parent_fleet: Fleet };\n\t"list_children" if "viewer" on "parent_fleet";\n\t"create_child" if "admin" on "parent_fleet";\n}\nhas_relation(fleet: Fleet, "parent_fleet", collection: UpdateTrustRootList)\n\tif collection.fleet = fleet;\n\n# Describes the policy for accessing blueprints\nresource TargetReleaseConfig {\n\tpermissions = [\n\t "read", # read the current target release\n\t "modify", # change the current target release\n\t];\n\n\trelations = { parent_fleet: Fleet };\n\t"read" if "viewer" on "parent_fleet";\n\t"modify" if "admin" on "parent_fleet";\n}\nhas_relation(fleet: Fleet, "parent_fleet", resource: TargetReleaseConfig)\n\tif resource.fleet = fleet;\n\n# Describes the policy for reading and modifying low-level inventory\nresource Inventory {\n\tpermissions = [ "read", "modify" ];\n\trelations = { parent_fleet: Fleet };\n\t"read" if "viewer" on "parent_fleet";\n\t"modify" if "admin" on "parent_fleet";\n}\nhas_relation(fleet: Fleet, "parent_fleet", inventory: Inventory)\n\tif inventory.fleet = fleet;\n\n# Describes the policy for accessing "/v1/system/ip-pools" in the API\nresource IpPoolList {\n\tpermissions = [\n\t "list_children",\n\t "modify",\n\t "create_child",\n\t];\n\n\t# Fleet Administrators can create or modify the IP Pools list.\n\trelations = { parent_fleet: Fleet };\n\t"modify" if "admin" on "parent_fleet";\n\t"create_child" if "admin" on "parent_fleet";\n\n\t# Fleet Viewers can list IP Pools\n\t"list_children" if "viewer" on "parent_fleet";\n}\nhas_relation(fleet: Fleet, "parent_fleet", ip_pool_list: IpPoolList)\n\tif ip_pool_list.fleet = fleet;\n\n# Any authenticated user can create a child of a provided IP Pool.\n# This is necessary to use the pools when provisioning instances.\nhas_permission(actor: AuthenticatedActor, "create_child", ip_pool: IpPool)\n\tif silo in actor.silo and silo.fleet = ip_pool.fleet;\n\n# Describes the policy for reading and writing the audit log \nresource AuditLog {\n\tpermissions = [\n\t "list_children", # retrieve audit log\n\t "create_child", # create audit log entry\n\t];\n\n\trelations = { parent_fleet: Fleet };\n\n\t# Fleet viewers can read the audit log\n\t"list_children" if "viewer" on "parent_fleet";\n}\n\n# Any actor should be able to write to the audit log because we need to be able\n# to write to the log from any request, authenticated or not. Audit log writes\n# are always a byproduct of other operations: there are no endpoints that allow\n# the user to write to the log deliberately. Note we use AuthenticatedActor\n# because we don't really mean unauthenticated -- in the case of login\n# operations, we use the external authenticator actor that creates the session\n# to authorize the audit log write.\nhas_permission(_actor: AuthenticatedActor, "create_child", _audit_log: AuditLog);\n\nhas_relation(fleet: Fleet, "parent_fleet", audit_log: AuditLog)\n\tif audit_log.fleet = fleet;\n\n# Describes the policy for creating and managing web console sessions.\nresource ConsoleSessionList {\n\tpermissions = [ "create_child" ];\n\trelations = { parent_fleet: Fleet };\n\t"create_child" if "external-authenticator" on "parent_fleet";\n}\nhas_relation(fleet: Fleet, "parent_fleet", collection: ConsoleSessionList)\n\tif collection.fleet = fleet;\n\n# Allow silo admins to delete and list user sessions\nresource SiloUserSessionList {\n permissions = [ "modify", "list_children" ];\n relations = { parent_silo: Silo };\n\n # A silo admin can modify (e.g., delete) a user's sessions.\n "modify" if "admin" on "parent_silo";\n\n # A silo admin can list a user's sessions.\n "list_children" if "admin" on "parent_silo";\n}\nhas_relation(silo: Silo, "parent_silo", authn_list: SiloUserSessionList)\n if authn_list.silo_user.silo = silo;\n\n# give users 'modify' and 'list_children' on their own sessions\nhas_permission(actor: AuthenticatedActor, "modify", authn_list: SiloUserSessionList)\n if actor.equals_silo_user(authn_list.silo_user);\nhas_permission(actor: AuthenticatedActor, "list_children", authn_list: SiloUserSessionList)\n if actor.equals_silo_user(authn_list.silo_user);\n\n# Allow silo admins to delete and list user access tokens\nresource SiloUserTokenList {\n permissions = [ "modify", "list_children" ];\n relations = { parent_silo: Silo };\n\n # A silo admin can modify (e.g., delete) a user's tokens.\n "modify" if "admin" on "parent_silo";\n\n # A silo admin can list a user's tokens.\n "list_children" if "admin" on "parent_silo";\n}\nhas_relation(silo: Silo, "parent_silo", authn_list: SiloUserTokenList)\n if authn_list.silo_user.silo = silo;\n\n# give users 'modify' and 'list_children' on their own tokens\nhas_permission(actor: AuthenticatedActor, "modify", authn_list: SiloUserTokenList)\n if actor.equals_silo_user(authn_list.silo_user);\nhas_permission(actor: AuthenticatedActor, "list_children", authn_list: SiloUserTokenList)\n if actor.equals_silo_user(authn_list.silo_user);\n\n# Describes the policy for creating and managing device authorization requests.\nresource DeviceAuthRequestList {\n\tpermissions = [ "create_child" ];\n\trelations = { parent_fleet: Fleet };\n\t"create_child" if "external-authenticator" on "parent_fleet";\n}\nhas_relation(fleet: Fleet, "parent_fleet", collection: DeviceAuthRequestList)\n\tif collection.fleet = fleet;\n\n# Describes the policy for creating and managing Silo certificates\nresource SiloCertificateList {\n\tpermissions = [ "list_children", "create_child" ];\n\n\trelations = { parent_silo: Silo, parent_fleet: Fleet };\n\n\t# Both Fleet and Silo administrators can see and modify the Silo's\n\t# certificates.\n\t"list_children" if "admin" on "parent_silo";\n\t"list_children" if "admin" on "parent_fleet";\n\t"create_child" if "admin" on "parent_silo";\n\t"create_child" if "admin" on "parent_fleet";\n}\nhas_relation(silo: Silo, "parent_silo", collection: SiloCertificateList)\n\tif collection.silo = silo;\nhas_relation(fleet: Fleet, "parent_fleet", collection: SiloCertificateList)\n\tif collection.silo.fleet = fleet;\n\n# Describes the policy for creating and managing Silo identity providers\nresource SiloIdentityProviderList {\n\tpermissions = [ "list_children", "create_child" ];\n\n\trelations = { parent_silo: Silo, parent_fleet: Fleet };\n\n\t# Everyone who can read the Silo (which includes all the users in the\n\t# Silo) can see the identity providers in it.\n\t"list_children" if "read" on "parent_silo";\n\n\t# Fleet and Silo administrators can manage the Silo's identity provider\n\t# configuration. This is one of the only areas of Silo configuration\n\t# that Fleet Administrators have permissions on. This is also one of\n\t# the only cases where we need to look two levels up the hierarchy to\n\t# see if somebody has the right permission. For most other things,\n\t# permissions cascade down the hierarchy so we only need to look at the\n\t# parent.\n\t"create_child" if "admin" on "parent_silo";\n\t"create_child" if "admin" on "parent_fleet";\n}\nhas_relation(silo: Silo, "parent_silo", collection: SiloIdentityProviderList)\n\tif collection.silo = silo;\nhas_relation(fleet: Fleet, "parent_fleet", collection: SiloIdentityProviderList)\n\tif collection.silo.fleet = fleet;\n\n# Describes the policy for creating and managing Silo users (mostly intended for\n# API-managed users)\nresource SiloUserList {\n\tpermissions = [ "list_children", "create_child" ];\n\n\trelations = { parent_silo: Silo, parent_fleet: Fleet };\n\n\t# Everyone who can read the Silo (which includes all the users in the\n\t# Silo) can see the users in it.\n\t"list_children" if "read" on "parent_silo";\n\n\t# Fleet and Silo administrators can manage the Silo's users. This is\n\t# one of the only areas of Silo configuration that Fleet Administrators\n\t# have permissions on. This is also one of the few cases (so far) where\n\t# we need to look two levels up the hierarchy to see if somebody has the\n\t# right permission. For most other things, permissions cascade down the\n\t# hierarchy so we only need to look at the parent.\n\t"create_child" if "admin" on "parent_silo";\n\t"list_children" if "admin" on "parent_fleet";\n\t"create_child" if "admin" on "parent_fleet";\n}\nhas_relation(silo: Silo, "parent_silo", collection: SiloUserList)\n\tif collection.silo = silo;\nhas_relation(fleet: Fleet, "parent_fleet", collection: SiloUserList)\n\tif collection.silo.fleet = fleet;\n\n# These rules grants the external authenticator role the permissions it needs to\n# read silo users and modify their sessions. This is necessary for login to\n# work.\nhas_permission(actor: AuthenticatedActor, "read", silo: Silo)\n\tif has_role(actor, "external-authenticator", silo.fleet);\nhas_permission(actor: AuthenticatedActor, "read", user: SiloUser)\n\tif has_role(actor, "external-authenticator", user.silo.fleet);\nhas_permission(actor: AuthenticatedActor, "modify", user: SiloUser)\n\tif has_role(actor, "external-authenticator", user.silo.fleet);\nhas_permission(actor: AuthenticatedActor, "read", group: SiloGroup)\n\tif has_role(actor, "external-authenticator", group.silo.fleet);\nhas_permission(actor: AuthenticatedActor, "modify", group: SiloGroup)\n\tif has_role(actor, "external-authenticator", group.silo.fleet);\n\nhas_permission(actor: AuthenticatedActor, "read", session: ConsoleSession)\n\tif has_role(actor, "external-authenticator", session.fleet);\nhas_permission(actor: AuthenticatedActor, "modify", session: ConsoleSession)\n\tif has_role(actor, "external-authenticator", session.fleet);\n\n# All authenticated users can read and delete device authn requests because\n# by necessity these operations happen before we've figured out what user (or\n# even Silo) the device auth is associated with. Any user can claim a device\n# auth request with the right user code (that's how it works) -- it's the user\n# code and associated logic that prevents unauthorized access here.\nhas_permission(_actor: AuthenticatedActor, "read", _device_auth: DeviceAuthRequest);\nhas_permission(_actor: AuthenticatedActor, "modify", _device_auth: DeviceAuthRequest);\n\nhas_permission(actor: AuthenticatedActor, "read", device_token: DeviceAccessToken)\n\tif has_role(actor, "external-authenticator", device_token.fleet);\n\nhas_permission(actor: AuthenticatedActor, "read", identity_provider: IdentityProvider)\n\tif has_role(actor, "external-authenticator", identity_provider.silo.fleet);\n\nhas_permission(actor: AuthenticatedActor, "read", saml_identity_provider: SamlIdentityProvider)\n\tif has_role(actor, "external-authenticator", saml_identity_provider.silo.fleet);\n\n# Describes the policy for who can access the internal database.\nresource Database {\n\tpermissions = [\n\t # "query" is required to perform any query against the database,\n\t # whether a read or write query. This is checked when an operation\n\t # checks out a database connection from the connection pool.\n\t #\n\t # Any authenticated user gets this permission. There's generally\n\t # some other authz check involved in the database query. For\n\t # example, if you're querying the database to "read" a "Project", we\n\t # should also be checking that. So why do we do this at all? It's\n\t # a belt-and-suspenders measure so that if we somehow introduced an\n\t # unauthenticated code path that hits the database, it cannot be\n\t # used to DoS the database because we won't allow the operation to\n\t # make the query. (As long as the code path _is_ authenticated, we\n\t # can use throttling mechanisms to prevent DoS.)\n\t "query",\n\n\t # "modify" is required to populate database data that's delivered\n\t # with the system. It should also be required for schema changes,\n\t # when we support those. This is separate from "query" so that we\n\t # cannot accidentally invoke these code paths from API calls and\n\t # other general functions.\n\t "modify"\n\t];\n}\n\n# All authenticated users have the "query" permission on the database.\nhas_permission(_actor: AuthenticatedActor, "query", _resource: Database);\n\n# The "db-init" user is the only one with the "modify" permission.\nhas_permission(USER_DB_INIT: AuthenticatedActor, "modify", _resource: Database);\nhas_permission(USER_DB_INIT: AuthenticatedActor, "create_child", _resource: IpPoolList);\n# It also has "admin" on the internal silo to populate it with built-in resources.\n# TODO-completeness: actually limit to just internal silo and not all silos\nhas_role(USER_DB_INIT: AuthenticatedActor, "admin", _silo: Silo);\n\n# Allow the internal API admin permissions on all silos.\nhas_role(USER_INTERNAL_API: AuthenticatedActor, "admin", _silo: Silo);\n\nresource WebhookSecret {\n\tpermissions = [ "read", "modify" ];\n\trelations = { parent_alert_receiver: AlertReceiver };\n\n\t"read" if "read" on "parent_alert_receiver";\n\t"modify" if "modify" on "parent_alert_receiver";\n}\n\nhas_relation(rx: AlertReceiver, "parent_alert_receiver", secret: WebhookSecret)\n\tif secret.alert_receiver = rx;\n\nresource AlertClassList {\n\tpermissions = [ "list_children" ];\n\trelations = { parent_fleet: Fleet };\n\n\t"list_children" if "viewer" on "parent_fleet";\n}\n\nhas_relation(fleet: Fleet, "parent_fleet", collection: AlertClassList)\n\tif collection.fleet = fleet;\n\n\n\n resource Disk {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_project: Project };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(parent: Project, "containing_project", child: Disk)\n if child.project = parent;\n \n\n resource Snapshot {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_project: Project };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(parent: Project, "containing_project", child: Snapshot)\n if child.project = parent;\n \n\n resource ProjectImage {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_project: Project };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(parent: Project, "containing_project", child: ProjectImage)\n if child.project = parent;\n \n\n resource AffinityGroup {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_project: Project };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(parent: Project, "containing_project", child: AffinityGroup)\n if child.project = parent;\n \n\n resource AntiAffinityGroup {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_project: Project };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(parent: Project, "containing_project", child: AntiAffinityGroup)\n if child.project = parent;\n \n\n resource Instance {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_project: Project };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(parent: Project, "containing_project", child: Instance)\n if child.project = parent;\n \n\n resource IpPool {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: IpPool)\n if child.fleet = fleet;\n \n\n resource InstanceNetworkInterface {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = {\n containing_project: Project,\n parent: Instance\n };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(project: Project, "containing_project", child: InstanceNetworkInterface)\n if has_relation(project, "containing_project", child.instance);\n\n has_relation(parent: Instance, "parent", child: InstanceNetworkInterface)\n if child.instance = parent;\n \n\n resource Vpc {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_project: Project };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(parent: Project, "containing_project", child: Vpc)\n if child.project = parent;\n \n\n resource VpcRouter {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = {\n containing_project: Project,\n parent: Vpc\n };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(project: Project, "containing_project", child: VpcRouter)\n if has_relation(project, "containing_project", child.vpc);\n\n has_relation(parent: Vpc, "parent", child: VpcRouter)\n if child.vpc = parent;\n \n\n resource InternetGateway {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = {\n containing_project: Project,\n parent: Vpc\n };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(project: Project, "containing_project", child: InternetGateway)\n if has_relation(project, "containing_project", child.vpc);\n\n has_relation(parent: Vpc, "parent", child: InternetGateway)\n if child.vpc = parent;\n \n\n resource InternetGatewayIpPool {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = {\n containing_project: Project,\n parent: InternetGateway\n };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(project: Project, "containing_project", child: InternetGatewayIpPool)\n if has_relation(project, "containing_project", child.internet_gateway);\n\n has_relation(parent: InternetGateway, "parent", child: InternetGatewayIpPool)\n if child.internet_gateway = parent;\n \n\n resource InternetGatewayIpAddress {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = {\n containing_project: Project,\n parent: InternetGateway\n };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(project: Project, "containing_project", child: InternetGatewayIpAddress)\n if has_relation(project, "containing_project", child.internet_gateway);\n\n has_relation(parent: InternetGateway, "parent", child: InternetGatewayIpAddress)\n if child.internet_gateway = parent;\n \n\n resource RouterRoute {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = {\n containing_project: Project,\n parent: VpcRouter\n };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(project: Project, "containing_project", child: RouterRoute)\n if has_relation(project, "containing_project", child.vpc_router);\n\n has_relation(parent: VpcRouter, "parent", child: RouterRoute)\n if child.vpc_router = parent;\n \n\n resource VpcSubnet {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = {\n containing_project: Project,\n parent: Vpc\n };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(project: Project, "containing_project", child: VpcSubnet)\n if has_relation(project, "containing_project", child.vpc);\n\n has_relation(parent: Vpc, "parent", child: VpcSubnet)\n if child.vpc = parent;\n \n\n resource FloatingIp {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_project: Project };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(parent: Project, "containing_project", child: FloatingIp)\n if child.project = parent;\n \n\n resource Image {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_silo: Silo };\n "list_children" if "viewer" on "containing_silo";\n "read" if "viewer" on "containing_silo";\n "modify" if "collaborator" on "containing_silo";\n "create_child" if "collaborator" on "containing_silo";\n }\n\n has_relation(parent: Silo, "containing_silo", child: Image)\n if child.silo = parent;\n \n\n resource SiloImage {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_silo: Silo };\n "list_children" if "viewer" on "containing_silo";\n "read" if "viewer" on "containing_silo";\n "modify" if "collaborator" on "containing_silo";\n "create_child" if "collaborator" on "containing_silo";\n }\n\n has_relation(parent: Silo, "containing_silo", child: SiloImage)\n if child.silo = parent;\n \n\n resource AddressLot {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: AddressLot)\n if child.fleet = fleet;\n \n\n resource Blueprint {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: Blueprint)\n if child.fleet = fleet;\n \n\n resource LoopbackAddress {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: LoopbackAddress)\n if child.fleet = fleet;\n \n\n\n resource ConsoleSession {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: ConsoleSession)\n if child.fleet = fleet;\n \n\n resource DeviceAuthRequest {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: DeviceAuthRequest)\n if child.fleet = fleet;\n \n\n resource DeviceAccessToken {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: DeviceAccessToken)\n if child.fleet = fleet;\n \n\n resource PhysicalDisk {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: PhysicalDisk)\n if child.fleet = fleet;\n \n\n resource Rack {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: Rack)\n if child.fleet = fleet;\n \n\n\n\n\n\n resource SupportBundle {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: SupportBundle)\n if child.fleet = fleet;\n \n\n\n\n resource Sled {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: Sled)\n if child.fleet = fleet;\n \n\n resource TufRepo {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: TufRepo)\n if child.fleet = fleet;\n \n\n resource TufArtifact {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: TufArtifact)\n if child.fleet = fleet;\n \n\n resource TufTrustRoot {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: TufTrustRoot)\n if child.fleet = fleet;\n \n\n resource Alert {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: Alert)\n if child.fleet = fleet;\n \n\n resource AlertReceiver {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: AlertReceiver)\n if child.fleet = fleet;\n \n\n\n resource Zpool {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: Zpool)\n if child.fleet = fleet;\n \n\n resource Service {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: Service)\n if child.fleet = fleet;\n \n\n resource UserBuiltin {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: UserBuiltin)\n if child.fleet = fleet;\n file = nexus/auth/src/authz/oso_generic.rs:89
762025-08-26T21:45:58.584ZINFOnexus (ServerContext): Setting up resolver using DNS servers for subnet: Ipv6Subnet { net: Ipv6Net { addr: fd00:1122:3344::, width: 48 } } file = nexus/src/context.rs:223
772025-08-26T21:45:58.587ZINFOnexus (DnsResolver): new DNS resolver addresses = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = internal-dns/resolver/src/resolver.rs:144
782025-08-26T21:45:58.591ZINFOnexus (ServerContext): Setting up qorb database pool from DNS dns_addrs = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = nexus/src/context.rs:279
792025-08-26T21:45:58.603ZDEBGnexus (ServerContext): registered USDT probes
802025-08-26T21:45:58.840ZINFOnexus: Database schema version is up to date desired_version = 183.0.0 file = nexus/db-queries/src/db/datastore/db_metadata.rs:143 found_version = 183.0.0
812025-08-26T21:45:58.844ZINFOnexus (SEC): SEC running file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/steno-0.4.1/src/sec.rs:813 sec_id = 5d0959e5-47a6-4bf3-a52b-57950b9d900d
822025-08-26T21:45:58.925ZDEBGnexus (clickhouse-client): registered USDT probes id = b93e7ed4-a04b-46a3-9b00-64d9815626a0
832025-08-26T21:45:58.967ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
842025-08-26T21:45:58.995ZDEBGnexus (DataLoader): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
852025-08-26T21:45:58.998ZINFOnexus (MgsUpdateDriver): starting MgsUpdateDriver file = nexus/mgs-updates/src/driver.rs:146
862025-08-26T21:45:59.000ZDEBGnexus (DataLoader): attempting to create built-in users
872025-08-26T21:45:59.003ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
882025-08-26T21:45:59.003ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
892025-08-26T21:45:59.069ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = GET path = /bgtasks versions = all
902025-08-26T21:45:59.069ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = POST path = /bgtasks/activate versions = all
912025-08-26T21:45:59.069ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = GET path = /bgtasks/view/{bgtask_name} versions = all
922025-08-26T21:45:59.069ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = GET path = /clickhouse/policy versions = all
932025-08-26T21:45:59.069ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = POST path = /clickhouse/policy versions = all
942025-08-26T21:45:59.069ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = POST path = /crucible/0/upstairs/{upstairs_id}/downstairs/{downstairs_id}/stop-request versions = all
952025-08-26T21:45:59.069ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = POST path = /crucible/0/upstairs/{upstairs_id}/downstairs/{downstairs_id}/stopped versions = all
962025-08-26T21:45:59.069ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = POST path = /crucible/0/upstairs/{upstairs_id}/repair/{repair_id}/progress versions = all
972025-08-26T21:45:59.069ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = POST path = /crucible/0/upstairs/{upstairs_id}/repair-finish versions = all
982025-08-26T21:45:59.069ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = POST path = /crucible/0/upstairs/{upstairs_id}/repair-start versions = all
992025-08-26T21:45:59.069ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = POST path = /demo-saga versions = all
1002025-08-26T21:45:59.069ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = POST path = /demo-saga/{demo_saga_id}/complete versions = all
1012025-08-26T21:45:59.069ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = GET path = /deployment/blueprints/all versions = all
1022025-08-26T21:45:59.070ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = DELETE path = /deployment/blueprints/all/{blueprint_id} versions = all
1032025-08-26T21:45:59.070ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = GET path = /deployment/blueprints/all/{blueprint_id} versions = all
1042025-08-26T21:45:59.070ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = POST path = /deployment/blueprints/import versions = all
1052025-08-26T21:45:59.070ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = POST path = /deployment/blueprints/regenerate versions = all
1062025-08-26T21:45:59.070ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = GET path = /deployment/blueprints/target versions = all
1072025-08-26T21:45:59.070ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = POST path = /deployment/blueprints/target versions = all
1082025-08-26T21:45:59.070ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = PUT path = /deployment/blueprints/target/enabled versions = all
1092025-08-26T21:45:59.070ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = GET path = /deployment/chicken-switches versions = all
1102025-08-26T21:45:59.070ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = POST path = /deployment/chicken-switches versions = all
1112025-08-26T21:45:59.070ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = GET path = /deployment/chicken-switches/{version} versions = all
1122025-08-26T21:45:59.070ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = GET path = /deployment/update-status versions = all
1132025-08-26T21:45:59.070ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = POST path = /disk/{disk_id}/remove-read-only-parent versions = all
1142025-08-26T21:45:59.070ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = PUT path = /disks/{disk_id} versions = all
1152025-08-26T21:45:59.070ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = GET path = /experimental/v1/system/support-bundles versions = all
1162025-08-26T21:45:59.070ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = POST path = /experimental/v1/system/support-bundles versions = all
1172025-08-26T21:45:59.070ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = DELETE path = /experimental/v1/system/support-bundles/{bundle_id} versions = all
1182025-08-26T21:45:59.070ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = GET path = /experimental/v1/system/support-bundles/{bundle_id} versions = all
1192025-08-26T21:45:59.070ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = PUT path = /experimental/v1/system/support-bundles/{bundle_id} versions = all
1202025-08-26T21:45:59.070ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = GET path = /experimental/v1/system/support-bundles/{bundle_id}/download versions = all
1212025-08-26T21:45:59.070ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = HEAD path = /experimental/v1/system/support-bundles/{bundle_id}/download versions = all
1222025-08-26T21:45:59.070ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = GET path = /experimental/v1/system/support-bundles/{bundle_id}/download/{file} versions = all
1232025-08-26T21:45:59.071ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = HEAD path = /experimental/v1/system/support-bundles/{bundle_id}/download/{file} versions = all
1242025-08-26T21:45:59.071ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = GET path = /experimental/v1/system/support-bundles/{bundle_id}/index versions = all
1252025-08-26T21:45:59.071ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = POST path = /instances/{instance_id}/migrate versions = all
1262025-08-26T21:45:59.071ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = POST path = /metrics/collectors versions = all
1272025-08-26T21:45:59.071ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = GET path = /metrics/collectors/{collector_id}/producers versions = all
1282025-08-26T21:45:59.071ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = POST path = /metrics/producers versions = all
1292025-08-26T21:45:59.071ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = GET path = /mgs-updates versions = all
1302025-08-26T21:45:59.071ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = GET path = /nat/ipv4/changeset/{from_gen} versions = all
1312025-08-26T21:45:59.071ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = GET path = /oximeter/read-policy versions = all
1322025-08-26T21:45:59.071ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = POST path = /oximeter/read-policy versions = all
1332025-08-26T21:45:59.071ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = POST path = /physical-disk/expunge versions = all
1342025-08-26T21:45:59.071ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = GET path = /probes/{sled} versions = all
1352025-08-26T21:45:59.071ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = GET path = /quiesce versions = all
1362025-08-26T21:45:59.071ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = POST path = /quiesce versions = all
1372025-08-26T21:45:59.071ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = PUT path = /racks/{rack_id}/initialization-complete versions = all
1382025-08-26T21:45:59.071ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = GET path = /sagas versions = all
1392025-08-26T21:45:59.071ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = GET path = /sagas/{saga_id} versions = all
1402025-08-26T21:45:59.071ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = GET path = /sled-agents/{sled_id} versions = all
1412025-08-26T21:45:59.071ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = POST path = /sled-agents/{sled_id} versions = all
1422025-08-26T21:45:59.072ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = POST path = /sled-agents/{sled_id}/firewall-rules-update versions = all
1432025-08-26T21:45:59.072ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = POST path = /sleds/add versions = all
1442025-08-26T21:45:59.072ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = POST path = /sleds/expunge versions = all
1452025-08-26T21:45:59.072ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = GET path = /sleds/uninitialized versions = all
1462025-08-26T21:45:59.072ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = PUT path = /switch/{switch_id} versions = all
1472025-08-26T21:45:59.072ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = GET path = /v1/ping versions = all
1482025-08-26T21:45:59.072ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = PUT path = /vmms/{propolis_id} versions = all
1492025-08-26T21:45:59.072ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = POST path = /volume/{volume_id}/remove-read-only-parent versions = all
1502025-08-26T21:45:59.072ZINFOnexus (dropshot_internal): listening file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:328 local_addr = [fd00:1122:3344:101::c]:12221
1512025-08-26T21:45:59.072ZDEBGnexus (dropshot_internal): successfully registered DTrace USDT probes local_addr = [fd00:1122:3344:101::c]:12221
1522025-08-26T21:45:59.072ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
1532025-08-26T21:45:59.073ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
1542025-08-26T21:45:59.080ZWARNnexus: Cannot look up rack: Object (of type ById(e37a46f2-f789-4ca4-9841-f8c6dd705964)) not found: rack file = nexus/src/app/rack.rs:781
1552025-08-26T21:45:59.159ZINFOnexus (DataLoader): created 0 built-in users file = nexus/db-queries/src/db/datastore/silo_user.rs:393
1562025-08-26T21:45:59.159ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
1572025-08-26T21:45:59.161ZDEBGnexus (DataLoader): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1582025-08-26T21:45:59.161ZDEBGnexus (DataLoader): attempting to create built-in role assignments
1592025-08-26T21:45:59.161ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
1602025-08-26T21:45:59.161ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1612025-08-26T21:46:00.050ZINFOnexus (DataLoader): created 0 built-in role assignments file = nexus/db-queries/src/db/datastore/role.rs:55
1622025-08-26T21:46:00.050ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
1632025-08-26T21:46:00.051ZDEBGnexus (DataLoader): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1642025-08-26T21:46:00.051ZDEBGnexus (DataLoader): attempting to create built-in silos
1652025-08-26T21:46:00.051ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
1662025-08-26T21:46:00.052ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1672025-08-26T21:46:00.235ZINFOnexus (DataLoader): created 0 built-in silos file = nexus/db-queries/src/db/datastore/silo.rs:101
1682025-08-26T21:46:00.237ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
1692025-08-26T21:46:00.238ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1702025-08-26T21:46:00.274ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
1712025-08-26T21:46:00.275ZDEBGnexus (DataLoader): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1722025-08-26T21:46:00.275ZDEBGnexus (DataLoader): attempting to create built-in projects
1732025-08-26T21:46:00.275ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
1742025-08-26T21:46:00.275ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1752025-08-26T21:46:00.337ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
1762025-08-26T21:46:00.338ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1772025-08-26T21:46:00.414ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
1782025-08-26T21:46:00.415ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1792025-08-26T21:46:00.418ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
1802025-08-26T21:46:00.420ZDEBGnexus (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) } result = Ok(())
1812025-08-26T21:46:00.420ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
1822025-08-26T21:46:00.420ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1832025-08-26T21:46:01.081ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
1842025-08-26T21:46:01.081ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
1852025-08-26T21:46:01.082ZWARNnexus: Cannot look up rack: Object (of type ById(e37a46f2-f789-4ca4-9841-f8c6dd705964)) not found: rack file = nexus/src/app/rack.rs:781
1862025-08-26T21:46:01.212ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
1872025-08-26T21:46:01.212ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1882025-08-26T21:46:01.214ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
1892025-08-26T21:46:01.215ZDEBGnexus (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) } result = Ok(())
1902025-08-26T21:46:01.215ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
1912025-08-26T21:46:01.216ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1922025-08-26T21:46:01.309ZINFOnexus (DataLoader): created built-in services project file = nexus/db-queries/src/db/datastore/project.rs:122
1932025-08-26T21:46:01.309ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
1942025-08-26T21:46:01.311ZDEBGnexus (DataLoader): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1952025-08-26T21:46:01.311ZDEBGnexus (DataLoader): attempting to create built-in VPCs
1962025-08-26T21:46:01.311ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
1972025-08-26T21:46:01.311ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1982025-08-26T21:46:01.322ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
1992025-08-26T21:46:01.323ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2002025-08-26T21:46:01.369ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2012025-08-26T21:46:01.370ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2022025-08-26T21:46:01.372ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2032025-08-26T21:46:01.372ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2042025-08-26T21:46:01.374ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2052025-08-26T21:46:01.374ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2062025-08-26T21:46:01.375ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2072025-08-26T21:46:01.378ZDEBGnexus (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) } result = Ok(())
2082025-08-26T21:46:01.378ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2092025-08-26T21:46:01.378ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2102025-08-26T21:46:01.417ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2112025-08-26T21:46:01.418ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2122025-08-26T21:46:01.459ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2132025-08-26T21:46:01.460ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2142025-08-26T21:46:01.461ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2152025-08-26T21:46:01.464ZDEBGnexus (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) } result = Ok(())
2162025-08-26T21:46:01.464ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2172025-08-26T21:46:01.464ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2182025-08-26T21:46:01.515ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2192025-08-26T21:46:01.516ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2202025-08-26T21:46:01.518ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2212025-08-26T21:46:01.518ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2222025-08-26T21:46:01.519ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2232025-08-26T21:46:01.519ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2242025-08-26T21:46:01.521ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2252025-08-26T21:46:01.524ZDEBGnexus (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") } result = Ok(())
2262025-08-26T21:46:01.524ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2272025-08-26T21:46:01.524ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2282025-08-26T21:46:01.537ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2292025-08-26T21:46:01.537ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2302025-08-26T21:46:01.554ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2312025-08-26T21:46:01.554ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2322025-08-26T21:46:01.571ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2332025-08-26T21:46:01.571ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2342025-08-26T21:46:01.573ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2352025-08-26T21:46:01.573ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2362025-08-26T21:46:01.579ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2372025-08-26T21:46:01.579ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2382025-08-26T21:46:01.581ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2392025-08-26T21:46:01.585ZDEBGnexus (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") } result = Ok(())
2402025-08-26T21:46:01.585ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2412025-08-26T21:46:01.585ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2422025-08-26T21:46:01.636ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2432025-08-26T21:46:01.636ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2442025-08-26T21:46:01.639ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2452025-08-26T21:46:01.639ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2462025-08-26T21:46:01.641ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2472025-08-26T21:46:01.641ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2482025-08-26T21:46:01.642ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2492025-08-26T21:46:01.646ZDEBGnexus (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = VpcRouter { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-074c-4000-8000-000000000001, lookup_type: ById(001de000-074c-4000-8000-000000000001) } result = Ok(())
2502025-08-26T21:46:01.646ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2512025-08-26T21:46:01.646ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2522025-08-26T21:46:01.841ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2532025-08-26T21:46:01.841ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2542025-08-26T21:46:01.848ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2552025-08-26T21:46:01.849ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2562025-08-26T21:46:01.854ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2572025-08-26T21:46:01.854ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2582025-08-26T21:46:01.857ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2592025-08-26T21:46:01.863ZDEBGnexus (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = VpcRouter { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-074c-4000-8000-000000000001, lookup_type: ById(001de000-074c-4000-8000-000000000001) } result = Ok(())
2602025-08-26T21:46:01.863ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2612025-08-26T21:46:01.863ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2622025-08-26T21:46:02.042ZDEBGnexus (DataLoader): attempting to create built-in VPC firewall rules
2632025-08-26T21:46:02.042ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2642025-08-26T21:46:02.042ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2652025-08-26T21:46:02.059ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2662025-08-26T21:46:02.060ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2672025-08-26T21:46:02.062ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2682025-08-26T21:46:02.062ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2692025-08-26T21:46:02.065ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2702025-08-26T21:46:02.065ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2712025-08-26T21:46:02.067ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2722025-08-26T21:46:02.068ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2732025-08-26T21:46:02.076ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2742025-08-26T21:46:02.077ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2752025-08-26T21:46:02.078ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2762025-08-26T21:46:02.082ZDEBGnexus (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
2772025-08-26T21:46:02.082ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2782025-08-26T21:46:02.082ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2792025-08-26T21:46:02.084ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2802025-08-26T21:46:02.084ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2812025-08-26T21:46:02.086ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2822025-08-26T21:46:02.086ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2832025-08-26T21:46:02.087ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2842025-08-26T21:46:02.091ZDEBGnexus (DataLoader): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
2852025-08-26T21:46:02.091ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2862025-08-26T21:46:02.091ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2872025-08-26T21:46:02.106ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2882025-08-26T21:46:02.106ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2892025-08-26T21:46:02.108ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2902025-08-26T21:46:02.108ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2912025-08-26T21:46:02.112ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2922025-08-26T21:46:02.112ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2932025-08-26T21:46:02.114ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2942025-08-26T21:46:02.117ZDEBGnexus (DataLoader): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
2952025-08-26T21:46:02.117ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2962025-08-26T21:46:02.117ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2972025-08-26T21:46:02.222ZINFOnexus (DataLoader): created built-in VPC firewall rules file = nexus/db-queries/src/db/datastore/vpc.rs:298
2982025-08-26T21:46:02.222ZDEBGnexus (DataLoader): attempting to create built-in VPC Subnets
2992025-08-26T21:46:02.222ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3002025-08-26T21:46:02.222ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3012025-08-26T21:46:02.234ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3022025-08-26T21:46:02.235ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3032025-08-26T21:46:02.237ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3042025-08-26T21:46:02.237ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3052025-08-26T21:46:02.240ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3062025-08-26T21:46:02.240ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3072025-08-26T21:46:02.242ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3082025-08-26T21:46:02.242ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3092025-08-26T21:46:02.244ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3102025-08-26T21:46:02.244ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3112025-08-26T21:46:02.246ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3122025-08-26T21:46:02.250ZDEBGnexus (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
3132025-08-26T21:46:02.250ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3142025-08-26T21:46:02.251ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3152025-08-26T21:46:02.264ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3162025-08-26T21:46:02.264ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3172025-08-26T21:46:02.266ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3182025-08-26T21:46:02.267ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3192025-08-26T21:46:02.268ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3202025-08-26T21:46:02.269ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3212025-08-26T21:46:02.270ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3222025-08-26T21:46:02.273ZDEBGnexus (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
3232025-08-26T21:46:02.295ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3242025-08-26T21:46:02.296ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3252025-08-26T21:46:02.297ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3262025-08-26T21:46:02.298ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3272025-08-26T21:46:02.300ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3282025-08-26T21:46:02.300ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3292025-08-26T21:46:02.302ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3302025-08-26T21:46:02.307ZDEBGnexus (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = VpcRouter { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-074c-4000-8000-000000000001, lookup_type: ById(001de000-074c-4000-8000-000000000001) } result = Ok(())
3312025-08-26T21:46:02.307ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3322025-08-26T21:46:02.307ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3332025-08-26T21:46:02.387ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3342025-08-26T21:46:02.387ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3352025-08-26T21:46:02.390ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3362025-08-26T21:46:02.390ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3372025-08-26T21:46:02.392ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3382025-08-26T21:46:02.392ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3392025-08-26T21:46:02.394ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3402025-08-26T21:46:02.394ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3412025-08-26T21:46:02.396ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3422025-08-26T21:46:02.399ZDEBGnexus (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
3432025-08-26T21:46:02.431ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3442025-08-26T21:46:02.431ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3452025-08-26T21:46:02.433ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3462025-08-26T21:46:02.433ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3472025-08-26T21:46:02.435ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3482025-08-26T21:46:02.435ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3492025-08-26T21:46:02.437ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3502025-08-26T21:46:02.440ZDEBGnexus (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = VpcRouter { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-074c-4000-8000-000000000001, lookup_type: ById(001de000-074c-4000-8000-000000000001) } result = Ok(())
3512025-08-26T21:46:02.440ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3522025-08-26T21:46:02.441ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3532025-08-26T21:46:02.500ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3542025-08-26T21:46:02.501ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3552025-08-26T21:46:02.503ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3562025-08-26T21:46:02.503ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3572025-08-26T21:46:02.504ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3582025-08-26T21:46:02.504ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3592025-08-26T21:46:02.506ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3602025-08-26T21:46:02.507ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3612025-08-26T21:46:02.508ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3622025-08-26T21:46:02.511ZDEBGnexus (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
3632025-08-26T21:46:02.543ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3642025-08-26T21:46:02.543ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3652025-08-26T21:46:02.544ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3662025-08-26T21:46:02.545ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3672025-08-26T21:46:02.546ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3682025-08-26T21:46:02.546ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3692025-08-26T21:46:02.553ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3702025-08-26T21:46:02.556ZDEBGnexus (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = VpcRouter { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-074c-4000-8000-000000000001, lookup_type: ById(001de000-074c-4000-8000-000000000001) } result = Ok(())
3712025-08-26T21:46:02.556ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3722025-08-26T21:46:02.556ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3732025-08-26T21:46:02.620ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3742025-08-26T21:46:02.621ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3752025-08-26T21:46:02.629ZINFOnexus (DataLoader): created built-in services vpc subnets file = nexus/db-queries/src/db/datastore/vpc.rs:364
3762025-08-26T21:46:02.629ZINFOnexus (DataLoader): created built-in services vpc file = nexus/db-queries/src/db/datastore/vpc.rs:230
3772025-08-26T21:46:02.629ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3782025-08-26T21:46:02.630ZDEBGnexus (DataLoader): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3792025-08-26T21:46:02.630ZDEBGnexus (DataLoader): attempting to create silo users
3802025-08-26T21:46:02.630ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3812025-08-26T21:46:02.631ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3822025-08-26T21:46:02.644ZINFOnexus (DataLoader): created 2 silo users file = nexus/db-queries/src/db/datastore/silo_user.rs:418
3832025-08-26T21:46:02.644ZDEBGnexus (DataLoader): attempting to create silo user role assignments
3842025-08-26T21:46:02.644ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3852025-08-26T21:46:02.644ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3862025-08-26T21:46:02.652ZINFOnexus (DataLoader): created 2 silo user role assignments file = nexus/db-queries/src/db/datastore/silo_user.rs:445
3872025-08-26T21:46:02.652ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3882025-08-26T21:46:02.652ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3892025-08-26T21:46:02.666ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3902025-08-26T21:46:02.667ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3912025-08-26T21:46:02.674ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3922025-08-26T21:46:02.674ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3932025-08-26T21:46:02.683ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3942025-08-26T21:46:02.684ZDEBGnexus (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = IpPoolList result = Ok(())
3952025-08-26T21:46:02.684ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3962025-08-26T21:46:02.684ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3972025-08-26T21:46:02.733ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3982025-08-26T21:46:02.734ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3992025-08-26T21:46:02.735ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
4002025-08-26T21:46:02.736ZDEBGnexus (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = IpPoolList result = Ok(())
4012025-08-26T21:46:02.736ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
4022025-08-26T21:46:02.737ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4032025-08-26T21:46:02.804ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
4042025-08-26T21:46:02.805ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4052025-08-26T21:46:02.812ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
4062025-08-26T21:46:02.812ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4072025-08-26T21:46:02.814ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
4082025-08-26T21:46:02.815ZDEBGnexus (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = IpPoolList result = Ok(())
4092025-08-26T21:46:02.815ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
4102025-08-26T21:46:02.815ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4112025-08-26T21:46:02.819ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
4122025-08-26T21:46:02.819ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4132025-08-26T21:46:02.821ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
4142025-08-26T21:46:02.822ZDEBGnexus (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = IpPoolList result = Ok(())
4152025-08-26T21:46:02.822ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
4162025-08-26T21:46:02.822ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4172025-08-26T21:46:02.826ZINFOnexus: populate complete file = nexus/src/app/mod.rs:548
4182025-08-26T21:46:02.826ZINFOnexus: activating background tasks file = nexus/src/app/mod.rs:557
4192025-08-26T21:46:02.832ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 1 reason = Timeout
4202025-08-26T21:46:02.832ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 1 reason = Timeout
4212025-08-26T21:46:02.832ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 1 reason = Timeout
4222025-08-26T21:46:02.832ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 1 reason = Timeout
4232025-08-26T21:46:02.832ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 1 reason = Timeout
4242025-08-26T21:46:02.832ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 1 reason = Timeout
4252025-08-26T21:46:02.832ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
4262025-08-26T21:46:02.832ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
4272025-08-26T21:46:02.832ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 1 reason = Timeout
4282025-08-26T21:46:02.832ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 1 reason = Timeout
4292025-08-26T21:46:02.832ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 1 reason = Timeout
4302025-08-26T21:46:02.832ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
4312025-08-26T21:46:02.833ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 1 reason = Timeout
4322025-08-26T21:46:02.833ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 1 reason = Timeout
4332025-08-26T21:46:02.833ZWARNnexus (BackgroundTasks): DNS propagation: skipped background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:61 reason = no config nor servers
4342025-08-26T21:46:02.833ZWARNnexus (BackgroundTasks): DNS propagation: skipped background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:61 reason = no config nor servers
4352025-08-26T21:46:02.834ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 358.448\u{b5}s iteration = 1 status = {"error":"no config nor servers"}
4362025-08-26T21:46:02.834ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 372.458\u{b5}s iteration = 1 status = {"error":"no config nor servers"}
4372025-08-26T21:46:02.834ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
4382025-08-26T21:46:02.838ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
4392025-08-26T21:46:02.838ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
4402025-08-26T21:46:02.838ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
4412025-08-26T21:46:02.838ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
4422025-08-26T21:46:02.839ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
4432025-08-26T21:46:02.839ZDEBGnexus (BackgroundTasks): activating background_task = inventory_collection iteration = 1 reason = Timeout
4442025-08-26T21:46:02.839ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 1 reason = Timeout
4452025-08-26T21:46:02.839ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
4462025-08-26T21:46:02.841ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-08-26 21:36:02.841706828 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
4472025-08-26T21:46:02.841ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
4482025-08-26T21:46:02.842ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
4492025-08-26T21:46:02.849ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
4502025-08-26T21:46:02.850ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
4512025-08-26T21:46:02.850ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
4522025-08-26T21:46:02.850ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
4532025-08-26T21:46:02.850ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
4542025-08-26T21:46:02.852ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 19.386228ms iteration = 1 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
4552025-08-26T21:46:02.852ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("60abf731-31e1-4111-a3e2-65da89eaed87.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("84c849eb-0949-41d9-ad02-9a5509144564.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("84c849eb-0949-41d9-ad02-9a5509144564.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::8)) }], valid_until: Instant { tv_sec: 866, tv_nsec: 709495118 } })
4562025-08-26T21:46:02.852ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("852fd1a3-de9a-47e8-a667-5e8dab880ad3.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("c68f1501-294c-4874-92ca-8fc0546a5870.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("d741a72f-8745-4723-9e80-e1e72987266b.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("d741a72f-8745-4723-9e80-e1e72987266b.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:3::1)) }], valid_until: Instant { tv_sec: 866, tv_nsec: 709865807 } })
4572025-08-26T21:46:02.855ZINFOnexus (BackgroundTasks): found DNS servers (initial) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external dns_group = external file = nexus/src/app/background/tasks/dns_servers.rs:113
4582025-08-26T21:46:02.855ZINFOnexus (BackgroundTasks): found DNS servers (initial) addresses = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal dns_group = internal file = nexus/src/app/background/tasks/dns_servers.rs:113
4592025-08-26T21:46:02.855ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 2 reason = Dependency
4602025-08-26T21:46:02.855ZWARNnexus (BackgroundTasks): DNS propagation: skipped background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:69 reason = no config
4612025-08-26T21:46:02.855ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 2 reason = Dependency
4622025-08-26T21:46:02.855ZWARNnexus (BackgroundTasks): DNS propagation: skipped background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:69 reason = no config
4632025-08-26T21:46:02.855ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 22.813604ms iteration = 1 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
4642025-08-26T21:46:02.855ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 12.721\u{b5}s iteration = 2 status = {"error":"no config"}
4652025-08-26T21:46:02.858ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4662025-08-26T21:46:02.858ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 18.6\u{b5}s iteration = 2 status = {"error":"no config"}
4672025-08-26T21:46:02.858ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 22.802543ms iteration = 1 status = {"addresses":["[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:3::1]:5353"]}
4682025-08-26T21:46:02.858ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 1 reason = Timeout
4692025-08-26T21:46:02.858ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:82 reason = no blueprint
4702025-08-26T21:46:02.858ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 23.339636ms iteration = 1 status = {"error":"no blueprint"}
4712025-08-26T21:46:02.858ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
4722025-08-26T21:46:02.858ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
4732025-08-26T21:46:02.858ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
4742025-08-26T21:46:02.858ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
4752025-08-26T21:46:02.859ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 1 reason = Timeout
4762025-08-26T21:46:02.859ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 1 reason = Timeout
4772025-08-26T21:46:02.859ZDEBGnexus (BackgroundTasks): blueprint planning disabled, doing nothing background_task = blueprint_planner
4782025-08-26T21:46:02.859ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 1 reason = Timeout
4792025-08-26T21:46:02.859ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 1 reason = Timeout
4802025-08-26T21:46:02.859ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:53
4812025-08-26T21:46:02.859ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
4822025-08-26T21:46:02.859ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 1 reason = Timeout
4832025-08-26T21:46:02.859ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
4842025-08-26T21:46:02.859ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 1 reason = Timeout
4852025-08-26T21:46:02.859ZDEBGnexus (BackgroundTasks): activating background_task = crdb_node_id_collector iteration = 1 reason = Timeout
4862025-08-26T21:46:02.859ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = crdb_node_id_collector file = nexus/src/app/background/tasks/crdb_node_id_collector.rs:78 reason = no blueprint
4872025-08-26T21:46:02.859ZDEBGnexus (BackgroundTasks): activation complete background_task = crdb_node_id_collector elapsed = 7.451\u{b5}s iteration = 1 status = {"error":"no blueprint"}
4882025-08-26T21:46:02.859ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 1 reason = Timeout
4892025-08-26T21:46:02.859ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 1 reason = Timeout
4902025-08-26T21:46:02.859ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 110.562\u{b5}s iteration = 1 status = "Disabled"
4912025-08-26T21:46:02.859ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 1 reason = Timeout
4922025-08-26T21:46:02.859ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 1 reason = Timeout
4932025-08-26T21:46:02.859ZDEBGnexus (BackgroundTasks): activating background_task = service_firewall_rule_propagation iteration = 1 reason = Timeout
4942025-08-26T21:46:02.859ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
4952025-08-26T21:46:02.859ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 1 reason = Timeout
4962025-08-26T21:46:02.859ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
4972025-08-26T21:46:02.859ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 1 reason = Timeout
4982025-08-26T21:46:02.859ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 1 reason = Timeout
4992025-08-26T21:46:02.859ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 1 reason = Timeout
5002025-08-26T21:46:02.859ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
5012025-08-26T21:46:02.859ZDEBGnexus (BackgroundTasks): activating background_task = saga_recovery iteration = 1 reason = Timeout
5022025-08-26T21:46:02.859ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
5032025-08-26T21:46:02.859ZDEBGnexus (service-firewall-rule-progator): starting background task for service firewall rule propagation background_task = service_firewall_rule_propagation
5042025-08-26T21:46:02.859ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
5052025-08-26T21:46:02.859ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 1 reason = Timeout
5062025-08-26T21:46:02.862ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
5072025-08-26T21:46:02.862ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
5082025-08-26T21:46:02.862ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 1 reason = Timeout
5092025-08-26T21:46:02.862ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 1 reason = Timeout
5102025-08-26T21:46:02.862ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
5112025-08-26T21:46:02.862ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 1 reason = Timeout
5122025-08-26T21:46:02.862ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
5132025-08-26T21:46:02.862ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
5142025-08-26T21:46:02.862ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
5152025-08-26T21:46:02.862ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 1 reason = Timeout
5162025-08-26T21:46:02.862ZDEBGnexus (BackgroundTasks): activating background_task = tuf_artifact_replication iteration = 1 reason = Timeout
5172025-08-26T21:46:02.862ZDEBGnexus (BackgroundTasks): activating background_task = alert_dispatcher iteration = 1 reason = Timeout
5182025-08-26T21:46:02.862ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
5192025-08-26T21:46:02.862ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5202025-08-26T21:46:02.864ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
5212025-08-26T21:46:02.864ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 1 reason = Timeout
5222025-08-26T21:46:02.864ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 1 reason = Timeout
5232025-08-26T21:46:02.864ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 1 reason = Timeout
5242025-08-26T21:46:02.864ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
5252025-08-26T21:46:02.864ZDEBGnexus (BackgroundTasks): activating background_task = webhook_deliverator iteration = 1 reason = Timeout
5262025-08-26T21:46:02.864ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
5272025-08-26T21:46:02.864ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
5282025-08-26T21:46:02.864ZDEBGnexus (BackgroundTasks): roles background_task = webhook_deliverator roles = RoleSet { roles: {} }
5292025-08-26T21:46:02.864ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-08-26 21:36:02.841706828 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
5302025-08-26T21:46:02.864ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 32.255817ms iteration = 1 status = {"expiration":"2025-08-26T21:36:02.841706828Z","pruned":[]}
5312025-08-26T21:46:02.864ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 5.045004ms iteration = 1 status = {"disabled":true,"errors":[],"sps":[]}
5322025-08-26T21:46:02.867ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
5332025-08-26T21:46:02.867ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 866, tv_nsec: 727464944 } })
5342025-08-26T21:46:02.868ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1205
5352025-08-26T21:46:02.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
5362025-08-26T21:46:02.872ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
5372025-08-26T21:46:02.872ZINFOnexus (SagaQuiesceHandle): saga recovery pass starting file = nexus/types/src/quiesce.rs:319
5382025-08-26T21:46:02.872ZDEBGnexus (SagaRecoverer): listing candidate sagas for recovery
5392025-08-26T21:46:02.872ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
5402025-08-26T21:46:02.872ZDEBGnexus (SagaRecoverer): roles roles = RoleSet { roles: {} }
5412025-08-26T21:46:02.873ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 866, tv_nsec: 732098328 } })
5422025-08-26T21:46:02.873ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_rendezvous iteration = 1 reason = Timeout
5432025-08-26T21:46:02.873ZWARNnexus (BackgroundTasks): Blueprint rendezvous: skipped background_task = blueprint_rendezvous file = nexus/src/app/background/tasks/blueprint_rendezvous.rs:52 reason = no blueprint
5442025-08-26T21:46:02.873ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_rendezvous elapsed = 13.712329ms iteration = 1 status = {"error":"no blueprint"}
5452025-08-26T21:46:02.873ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1205
5462025-08-26T21:46:02.874ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
5472025-08-26T21:46:02.874ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
5482025-08-26T21:46:02.874ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
5492025-08-26T21:46:02.874ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
5502025-08-26T21:46:02.874ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
5512025-08-26T21:46:02.874ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
5522025-08-26T21:46:02.874ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
5532025-08-26T21:46:02.874ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
5542025-08-26T21:46:02.874ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5552025-08-26T21:46:02.874ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
5562025-08-26T21:46:02.874ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
5572025-08-26T21:46:02.874ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
5582025-08-26T21:46:02.875ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
5592025-08-26T21:46:02.875ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
5602025-08-26T21:46:02.875ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5612025-08-26T21:46:02.875ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = webhook_deliverator resource = Database result = Ok(())
5622025-08-26T21:46:02.875ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
5632025-08-26T21:46:02.877ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5642025-08-26T21:46:02.877ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
5652025-08-26T21:46:02.878ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
5662025-08-26T21:46:02.878ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 18.60624ms iteration = 1 status = {"errors":[],"found_port_ok":[]}
5672025-08-26T21:46:02.878ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
5682025-08-26T21:46:02.878ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5692025-08-26T21:46:02.878ZDEBGnexus (SagaRecoverer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000003a8a, .. }) resource = Database result = Ok(())
5702025-08-26T21:46:02.878ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = alert_dispatcher resource = Database result = Ok(())
5712025-08-26T21:46:02.878ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
5722025-08-26T21:46:02.878ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
5732025-08-26T21:46:02.878ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
5742025-08-26T21:46:02.878ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
5752025-08-26T21:46:02.878ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
5762025-08-26T21:46:02.879ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
5772025-08-26T21:46:02.879ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
5782025-08-26T21:46:02.879ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
5792025-08-26T21:46:02.879ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
5802025-08-26T21:46:02.880ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
5812025-08-26T21:46:02.880ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
5822025-08-26T21:46:02.882ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
5832025-08-26T21:46:02.883ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
5842025-08-26T21:46:02.883ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
5852025-08-26T21:46:02.884ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
5862025-08-26T21:46:02.884ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
5872025-08-26T21:46:02.884ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5882025-08-26T21:46:02.884ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5892025-08-26T21:46:02.884ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
5902025-08-26T21:46:02.884ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
5912025-08-26T21:46:02.885ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
5922025-08-26T21:46:02.901ZINFOnexus (BackgroundTasks): all webhook delivery tasks completed background_task = webhook_deliverator file = nexus/src/app/background/tasks/webhook_deliverator.rs:204 num_receivers = 0
5932025-08-26T21:46:02.901ZDEBGnexus (BackgroundTasks): activation complete background_task = webhook_deliverator elapsed = 41.648329ms iteration = 1 status = {"by_rx":{},"error":null}
5942025-08-26T21:46:02.919ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5952025-08-26T21:46:02.923ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
5962025-08-26T21:46:02.923ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5972025-08-26T21:46:02.925ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5982025-08-26T21:46:02.926ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5992025-08-26T21:46:02.926ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
6002025-08-26T21:46:02.926ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
6012025-08-26T21:46:02.926ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
6022025-08-26T21:46:02.927ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6032025-08-26T21:46:02.929ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
6042025-08-26T21:46:02.929ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
6052025-08-26T21:46:02.929ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
6062025-08-26T21:46:02.930ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
6072025-08-26T21:46:02.930ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
6082025-08-26T21:46:02.931ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
6092025-08-26T21:46:02.931ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
6102025-08-26T21:46:02.931ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
6112025-08-26T21:46:02.931ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
6122025-08-26T21:46:02.942ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
6132025-08-26T21:46:02.943ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = alert_dispatcher resource = Database result = Ok(())
6142025-08-26T21:46:02.948ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6152025-08-26T21:46:02.952ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
6162025-08-26T21:46:02.952ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6172025-08-26T21:46:02.952ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
6182025-08-26T21:46:02.952ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
6192025-08-26T21:46:02.952ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
6202025-08-26T21:46:02.953ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 93.209801ms iteration = 1 status = {"errors":[],"requests_created_ok":[]}
6212025-08-26T21:46:02.953ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
6222025-08-26T21:46:02.953ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 93.543589ms iteration = 1 status = {"errors":[],"finish_invoked_ok":[]}
6232025-08-26T21:46:02.953ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 94.179853ms iteration = 1 status = {"errors":[],"garbage_collect_requested":[]}
6242025-08-26T21:46:02.954ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6252025-08-26T21:46:02.955ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
6262025-08-26T21:46:02.955ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
6272025-08-26T21:46:02.956ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 97.022867ms iteration = 1 status = {}
6282025-08-26T21:46:02.956ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6292025-08-26T21:46:02.956ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
6302025-08-26T21:46:02.956ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
6312025-08-26T21:46:02.956ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
6322025-08-26T21:46:02.957ZINFOnexus (SagaRecoverer): listed in-progress sagas count = 0 file = nexus/src/app/background/tasks/saga_recovery.rs:473
6332025-08-26T21:46:02.957ZINFOnexus (SagaQuiesceHandle): saga recovery pass finished file = nexus/types/src/quiesce.rs:332 generation = 1
6342025-08-26T21:46:02.957ZDEBGnexus (BackgroundTasks): activation complete background_task = saga_recovery elapsed = 98.139842ms iteration = 1 status = {"last_pass":{"Success":{"nfailed":0,"nfound":0,"nrecovered":0,"nremoved":0,"nskipped":0}},"ntotal_failures":0,"ntotal_finished":0,"ntotal_recovered":0,"ntotal_sec_errors_bad_state":0,"ntotal_sec_errors_missing":0,"ntotal_started":0,"recent_failures":[],"recent_recoveries":[]}
6352025-08-26T21:46:02.957ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
6362025-08-26T21:46:02.957ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
6372025-08-26T21:46:02.957ZDEBGnexus (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = []
6382025-08-26T21:46:02.958ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
6392025-08-26T21:46:02.958ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 866, tv_nsec: 818218469 } })
6402025-08-26T21:46:02.958ZDEBGnexus (BackgroundTasks): activation complete background_task = alert_dispatcher elapsed = 98.91572ms iteration = 1 status = {"dispatched":[],"errors":[],"glob_version":"183.0.0","globs_reprocessed":{},"no_receivers":[]}
6412025-08-26T21:46:02.958ZINFOnexus (BackgroundTasks): Physical Disk Adoption: Rack not yet initialized background_task = physical_disk_adoption file = nexus/src/app/background/tasks/physical_disk_adoption.rs:81 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
6422025-08-26T21:46:02.958ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 99.33037ms iteration = 1 status = {"error":"rack not yet initialized: e37a46f2-f789-4ca4-9841-f8c6dd705964"}
6432025-08-26T21:46:02.959ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6442025-08-26T21:46:02.960ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
6452025-08-26T21:46:02.960ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
6462025-08-26T21:46:02.960ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
6472025-08-26T21:46:02.960ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6482025-08-26T21:46:02.964ZWARNnexus (BackgroundTasks): failed to read DNS config background_task = dns_config_internal dns_group = internal error = Internal Error: expected exactly one latest version for DNS group Internal, found 0 file = nexus/src/app/background/tasks/dns_config.rs:72
6492025-08-26T21:46:02.964ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 131.507554ms iteration = 1 status = {"error":"failed to read DNS config: Internal Error: expected exactly one latest version for DNS group Internal, found 0"}
6502025-08-26T21:46:02.964ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
6512025-08-26T21:46:02.964ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
6522025-08-26T21:46:02.964ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
6532025-08-26T21:46:02.965ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
6542025-08-26T21:46:02.965ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
6552025-08-26T21:46:02.965ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6562025-08-26T21:46:02.965ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
6572025-08-26T21:46:02.967ZWARNnexus (BackgroundTasks): failed to read target blueprint background_task = blueprint_loader error = Internal Error: no target blueprint set file = nexus/src/app/background/tasks/blueprint_load.rs:69
6582025-08-26T21:46:02.967ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 107.79913ms iteration = 1 status = {"error":"failed to read target blueprint: Internal Error: no target blueprint set"}
6592025-08-26T21:46:02.968ZWARNnexus (BackgroundTasks): inventory collection is None background_task = service_zone_nat_tracker file = nexus/src/app/background/tasks/sync_service_zone_nat.rs:91
6602025-08-26T21:46:02.968ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 108.716461ms iteration = 1 status = {"error":"inventory collection is None"}
6612025-08-26T21:46:02.969ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Fleet result = Ok(())
6622025-08-26T21:46:02.969ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
6632025-08-26T21:46:02.969ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1213 zone_address = fd00:1122:3344:101::2
6642025-08-26T21:46:02.969ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
6652025-08-26T21:46:02.969ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
6662025-08-26T21:46:02.971ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "c65952b2-1194-47b7-a735-17d245c9cb00", "content-length": "26", "date": "Tue, 26 Aug 2025 21:46:02 GMT"} })
6672025-08-26T21:46:02.972ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
6682025-08-26T21:46:02.972ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 132.498267ms iteration = 1 status = {"chicken_switches_updated":false}
6692025-08-26T21:46:02.972ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
6702025-08-26T21:46:02.973ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
6712025-08-26T21:46:02.973ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1213 zone_address = fd00:1122:3344:101::2
6722025-08-26T21:46:02.973ZDEBGnexus (MgsClient): client request background_task = nat_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
6732025-08-26T21:46:02.973ZDEBGnexus (MgsClient): client response background_task = nat_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "7a8f205c-f413-4de9-b475-ad068ce1e7c8", "content-length": "26", "date": "Tue, 26 Aug 2025 21:46:02 GMT"} })
6742025-08-26T21:46:02.975ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
6752025-08-26T21:46:02.975ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
6762025-08-26T21:46:02.977ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1216 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
6772025-08-26T21:46:02.977ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1216 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
6782025-08-26T21:46:02.978ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6792025-08-26T21:46:02.980ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Fleet result = Ok(())
6802025-08-26T21:46:02.980ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
6812025-08-26T21:46:02.980ZWARNnexus (BackgroundTasks): failed to read Silo/DNS/TLS configuration background_task = external_endpoints error = Internal Error: expected at least one external DNS zone file = nexus/src/app/background/tasks/external_endpoints.rs:48
6822025-08-26T21:46:02.980ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 147.976326ms iteration = 1 status = {"error":"failed to read Silo/DNS/TLS configuration: Internal Error: expected at least one external DNS zone"}
6832025-08-26T21:46:02.980ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
6842025-08-26T21:46:02.981ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
6852025-08-26T21:46:02.981ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
6862025-08-26T21:46:02.982ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
6872025-08-26T21:46:02.982ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
6882025-08-26T21:46:02.982ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
6892025-08-26T21:46:02.982ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
6902025-08-26T21:46:03.006ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
6912025-08-26T21:46:03.006ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
6922025-08-26T21:46:03.008ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
6932025-08-26T21:46:03.009ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
6942025-08-26T21:46:03.018ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
6952025-08-26T21:46:03.018ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
6962025-08-26T21:46:03.019ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
6972025-08-26T21:46:03.019ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
6982025-08-26T21:46:03.019ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:138 found = 0 not_ready_to_be_deleted = 0
6992025-08-26T21:46:03.019ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:170 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
7002025-08-26T21:46:03.019ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 159.961696ms iteration = 1 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
7012025-08-26T21:46:03.019ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 159.869583ms iteration = 1 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
7022025-08-26T21:46:03.020ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7032025-08-26T21:46:03.020ZWARNnexus (BackgroundTasks): failed to read DNS config background_task = dns_config_external dns_group = external error = Internal Error: expected exactly one latest version for DNS group External, found 0 file = nexus/src/app/background/tasks/dns_config.rs:72
7042025-08-26T21:46:03.020ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 188.015648ms iteration = 1 status = {"error":"failed to read DNS config: Internal Error: expected exactly one latest version for DNS group External, found 0"}
7052025-08-26T21:46:03.021ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
7062025-08-26T21:46:03.021ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
7072025-08-26T21:46:03.023ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
7082025-08-26T21:46:03.023ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
7092025-08-26T21:46:03.024ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
7102025-08-26T21:46:03.024ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
7112025-08-26T21:46:03.024ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
7122025-08-26T21:46:03.025ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7132025-08-26T21:46:03.029ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
7142025-08-26T21:46:03.029ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
7152025-08-26T21:46:03.029ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
7162025-08-26T21:46:03.031ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
7172025-08-26T21:46:03.031ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
7182025-08-26T21:46:03.031ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
7192025-08-26T21:46:03.032ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
7202025-08-26T21:46:03.032ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:550 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
7212025-08-26T21:46:03.032ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 173.019251ms iteration = 1 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
7222025-08-26T21:46:03.033ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
7232025-08-26T21:46:03.033ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
7242025-08-26T21:46:03.034ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 175.149488ms iteration = 1 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
7252025-08-26T21:46:03.035ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7262025-08-26T21:46:03.036ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _cockroach._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_cockroach._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("5362db44-dfc6-41c1-8ec8-32ab3e86b7b3.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("5e2b44ef-e7e6-47ec-9087-a2fb44127ede.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("70604151-9a25-4539-bcae-b8d33d91f6a5.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("be83bf79-a7c5-40a7-be80-31ec2f206604.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("d35c8171-47d8-4704-93d7-f07739a9281c.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("d35c8171-47d8-4704-93d7-f07739a9281c.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::7)) }], valid_until: Instant { tv_sec: 866, tv_nsec: 896385502 } })
7272025-08-26T21:46:03.040ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
7282025-08-26T21:46:03.040ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
7292025-08-26T21:46:03.040ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
7302025-08-26T21:46:03.044ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 184.849057ms iteration = 1 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
7312025-08-26T21:46:03.047ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
7322025-08-26T21:46:03.048ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
7332025-08-26T21:46:03.051ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
7342025-08-26T21:46:03.051ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
7352025-08-26T21:46:03.053ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
7362025-08-26T21:46:03.054ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
7372025-08-26T21:46:03.055ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
7382025-08-26T21:46:03.056ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
7392025-08-26T21:46:03.057ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
7402025-08-26T21:46:03.058ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
7412025-08-26T21:46:03.059ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 199.430445ms iteration = 1 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
7422025-08-26T21:46:03.059ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
7432025-08-26T21:46:03.060ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
7442025-08-26T21:46:03.061ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7452025-08-26T21:46:03.062ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
7462025-08-26T21:46:03.062ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
7472025-08-26T21:46:03.064ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
7482025-08-26T21:46:03.064ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
7492025-08-26T21:46:03.064ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
7502025-08-26T21:46:03.065ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
7512025-08-26T21:46:03.067ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
7522025-08-26T21:46:03.067ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
7532025-08-26T21:46:03.068ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
7542025-08-26T21:46:03.070ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
7552025-08-26T21:46:03.070ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
7562025-08-26T21:46:03.072ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
7572025-08-26T21:46:03.073ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
7582025-08-26T21:46:03.074ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:407
7592025-08-26T21:46:03.074ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 215.371566ms iteration = 1 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
7602025-08-26T21:46:03.075ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
7612025-08-26T21:46:03.076ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
7622025-08-26T21:46:03.078ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
7632025-08-26T21:46:03.079ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
7642025-08-26T21:46:03.079ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 246.562398ms iteration = 1 status = null
7652025-08-26T21:46:03.079ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 246.65299ms iteration = 1 status = {}
7662025-08-26T21:46:03.084ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
7672025-08-26T21:46:03.084ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
7682025-08-26T21:46:03.085ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
7692025-08-26T21:46:03.085ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
7702025-08-26T21:46:03.086ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
7712025-08-26T21:46:03.086ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
7722025-08-26T21:46:03.095ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 236.053681ms iteration = 1 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
7732025-08-26T21:46:03.105ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
7742025-08-26T21:46:03.105ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
7752025-08-26T21:46:03.132ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
7762025-08-26T21:46:03.132ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
7772025-08-26T21:46:03.132ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 272.919152ms iteration = 1 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
7782025-08-26T21:46:03.133ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
7792025-08-26T21:46:03.133ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
7802025-08-26T21:46:03.135ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
7812025-08-26T21:46:03.136ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
7822025-08-26T21:46:03.140ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7832025-08-26T21:46:03.144ZDEBGnexus (ServiceBalancer): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Rack { parent: Fleet, key: e37a46f2-f789-4ca4-9841-f8c6dd705964, lookup_type: ById(e37a46f2-f789-4ca4-9841-f8c6dd705964) } result = Ok(())
7842025-08-26T21:46:03.144ZINFOnexus: Still waiting for rack initialization: Rack { identity: RackIdentity { id: e37a46f2-f789-4ca4-9841-f8c6dd705964, time_created: 2025-08-26T21:46:02.666890Z, time_modified: 2025-08-26T21:46:02.666890Z }, initialized: false, tuf_base_url: None, rack_subnet: None } file = nexus/src/app/rack.rs:775
7852025-08-26T21:46:03.170ZINFOnexus (BackgroundTasks): VPC route manager sled dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
7862025-08-26T21:46:03.170ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
7872025-08-26T21:46:03.170ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
7882025-08-26T21:46:03.171ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
7892025-08-26T21:46:03.172ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "66824720-7626-46ab-97d0-4499f8d82671", "content-length": "312", "date": "Tue, 26 Aug 2025 21:46:02 GMT"} })
7902025-08-26T21:46:03.172ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7912025-08-26T21:46:03.175ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Fleet result = Ok(())
7922025-08-26T21:46:03.175ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
7932025-08-26T21:46:03.175ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
7942025-08-26T21:46:03.176ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
7952025-08-26T21:46:03.177ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
7962025-08-26T21:46:03.177ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
7972025-08-26T21:46:03.177ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
7982025-08-26T21:46:03.179ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7992025-08-26T21:46:03.181ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8002025-08-26T21:46:03.181ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Fleet result = Ok(())
8012025-08-26T21:46:03.181ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
8022025-08-26T21:46:03.182ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
8032025-08-26T21:46:03.186ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = tuf_artifact_replication body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/artifacts-config
8042025-08-26T21:46:03.187ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
8052025-08-26T21:46:03.187ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
8062025-08-26T21:46:03.187ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
8072025-08-26T21:46:03.188ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
8082025-08-26T21:46:03.189ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
8092025-08-26T21:46:03.191ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
8102025-08-26T21:46:03.192ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
8112025-08-26T21:46:03.193ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8122025-08-26T21:46:03.195ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = tuf_artifact_replication result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/artifacts-config", status: 204, headers: {"x-request-id": "0ed33e9a-3384-4281-a579-6738140f43e7", "date": "Tue, 26 Aug 2025 21:46:02 GMT"} })
8132025-08-26T21:46:03.195ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = tuf_artifact_replication body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/artifacts
8142025-08-26T21:46:03.196ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = tuf_artifact_replication result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/artifacts", status: 200, headers: {"content-type": "application/json", "x-request-id": "7c3282fa-c718-4ee3-8994-cbb0160c33ac", "content-length": "26", "date": "Tue, 26 Aug 2025 21:46:02 GMT"} })
8152025-08-26T21:46:03.196ZINFOnexus (BackgroundTasks): Successfully got artifact list background_task = tuf_artifact_replication file = nexus/src/app/background/tasks/tuf_artifact_replication.rs:672 sled = http://[fd00:1122:3344:101::1]:12345
8162025-08-26T21:46:03.200ZDEBGnexus (BackgroundTasks): activation complete background_task = tuf_artifact_replication elapsed = 340.376552ms iteration = 1 status = {"generation":1,"last_run_counters":{"copy_err":0,"copy_ok":0,"list_err":0,"list_ok":1,"put_config_err":0,"put_config_ok":1,"put_err":0,"put_ok":0},"lifetime_counters":{"copy_err":0,"copy_ok":0,"list_err":0,"list_ok":1,"put_config_err":0,"put_config_ok":1,"put_err":0,"put_ok":0},"local_repos":0,"request_debug_ringbuf":[{"generation":1,"operation":"put_config","target_sled":"dd0c655a-95c2-48fc-b4c3-9733ed4b0faa","time":"2025-08-26T21:46:03.195562885Z"},{"operation":"list","target_sled":"dd0c655a-95c2-48fc-b4c3-9733ed4b0faa","time":"2025-08-26T21:46:03.196425124Z"}]}
8172025-08-26T21:46:03.200ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
8182025-08-26T21:46:03.200ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
8192025-08-26T21:46:03.201ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
8202025-08-26T21:46:03.215ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
8212025-08-26T21:46:03.215ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
8222025-08-26T21:46:03.217ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
8232025-08-26T21:46:03.217ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
8242025-08-26T21:46:03.219ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
8252025-08-26T21:46:03.219ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
8262025-08-26T21:46:03.222ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
8272025-08-26T21:46:03.222ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
8282025-08-26T21:46:03.225ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
8292025-08-26T21:46:03.225ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
8302025-08-26T21:46:03.227ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
8312025-08-26T21:46:03.227ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
8322025-08-26T21:46:03.229ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
8332025-08-26T21:46:03.229ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
8342025-08-26T21:46:03.229ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
8352025-08-26T21:46:03.229ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
8362025-08-26T21:46:03.231ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8372025-08-26T21:46:03.231ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8382025-08-26T21:46:03.234ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
8392025-08-26T21:46:03.234ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
8402025-08-26T21:46:03.234ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
8412025-08-26T21:46:03.237ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
8422025-08-26T21:46:03.237ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
8432025-08-26T21:46:03.237ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
8442025-08-26T21:46:03.239ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
8452025-08-26T21:46:03.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
8462025-08-26T21:46:03.240ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
8472025-08-26T21:46:03.241ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
8482025-08-26T21:46:03.242ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8492025-08-26T21:46:03.248ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
8502025-08-26T21:46:03.248ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
8512025-08-26T21:46:03.248ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
8522025-08-26T21:46:03.256ZDEBGnexus (service-firewall-rule-progator): resolved names for firewall rules background_task = service_firewall_rule_propagation instance_interfaces = {} subnet_interfaces = {} subnet_networks = {Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })], Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })]} vpc_interfaces = {}
8532025-08-26T21:46:03.257ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
8542025-08-26T21:46:03.257ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
8552025-08-26T21:46:03.258ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8562025-08-26T21:46:03.260ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Fleet result = Ok(())
8572025-08-26T21:46:03.260ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
8582025-08-26T21:46:03.260ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
8592025-08-26T21:46:03.270ZDEBGnexus (service-firewall-rule-progator): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified. background_task = service_firewall_rule_propagation
8602025-08-26T21:46:03.270ZDEBGnexus (service-firewall-rule-progator): resolved firewall rules for sled agents background_task = service_firewall_rule_propagation sled_agent_rules = []
8612025-08-26T21:46:03.270ZDEBGnexus (service-firewall-rule-progator): resolved 0 rules for sleds background_task = service_firewall_rule_propagation
8622025-08-26T21:46:03.278ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8632025-08-26T21:46:03.280ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
8642025-08-26T21:46:03.280ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
8652025-08-26T21:46:03.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
8662025-08-26T21:46:03.294ZINFOnexus (BackgroundTasks): instance reincarnation completed successfully background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:113 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0
8672025-08-26T21:46:03.294ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 435.326124ms iteration = 1 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
8682025-08-26T21:46:03.303ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
8692025-08-26T21:46:03.303ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
8702025-08-26T21:46:03.304ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "dbfce80e-de6e-4e04-9d4e-f8964f372082", "date": "Tue, 26 Aug 2025 21:46:02 GMT"} })
8712025-08-26T21:46:03.304ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
8722025-08-26T21:46:03.305ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
8732025-08-26T21:46:03.307ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
8742025-08-26T21:46:03.307ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
8752025-08-26T21:46:03.308ZDEBGnexus (CockroachClusterAdminClient): Updated CockroachDB cluster backends added = 5 addresses = [[fd00:1122:3344:101::6]:32222, [fd00:1122:3344:101::5]:32222, [fd00:1122:3344:101::3]:32222, [fd00:1122:3344:101::4]:32222, [fd00:1122:3344:101::7]:32222] removed = 0 total = 5
8762025-08-26T21:46:03.310ZDEBGnexus (BackgroundTasks): begin collection background_task = inventory_collection
8772025-08-26T21:46:03.310ZDEBGnexus (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[fd00:1122:3344:101::2]:12225
8782025-08-26T21:46:03.310ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/ignition
8792025-08-26T21:46:03.311ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "8abf751a-b9ed-4f1e-bae0-b19e9860d459", "content-length": "476", "date": "Tue, 26 Aug 2025 21:46:03 GMT"} })
8802025-08-26T21:46:03.312ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
8812025-08-26T21:46:03.312ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
8822025-08-26T21:46:03.316ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
8832025-08-26T21:46:03.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
8842025-08-26T21:46:03.318ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0
8852025-08-26T21:46:03.318ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
8862025-08-26T21:46:03.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
8872025-08-26T21:46:03.319ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "3ac4b786-2b86-4ffc-bb7e-2bc8cb1c4e2f", "content-length": "734", "date": "Tue, 26 Aug 2025 21:46:03 GMT"} })
8882025-08-26T21:46:03.320ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0
8892025-08-26T21:46:03.320ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
8902025-08-26T21:46:03.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
8912025-08-26T21:46:03.321ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "3b3594e3-bdd1-4054-990b-102a667a44ca", "content-length": "111", "date": "Tue, 26 Aug 2025 21:46:03 GMT"} })
8922025-08-26T21:46:03.322ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
8932025-08-26T21:46:03.323ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
8942025-08-26T21:46:03.323ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1
8952025-08-26T21:46:03.324ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "52acccfb-1f6d-4dfb-a148-ebdbdaea1da7", "content-length": "111", "date": "Tue, 26 Aug 2025 21:46:03 GMT"} })
8962025-08-26T21:46:03.324ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0
8972025-08-26T21:46:03.324ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "f91798c6-1c53-48f0-960a-6dcb8fdf040f", "content-length": "163", "date": "Tue, 26 Aug 2025 21:46:03 GMT"} })
8982025-08-26T21:46:03.324ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1
8992025-08-26T21:46:03.325ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "7b358869-f272-4fab-b83f-8d220d75d3e0", "content-length": "163", "date": "Tue, 26 Aug 2025 21:46:03 GMT"} })
9002025-08-26T21:46:03.325ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0
9012025-08-26T21:46:03.325ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
9022025-08-26T21:46:03.325ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "b5704083-fd24-4c9f-9bdd-8eab8dc25e32", "content-length": "166", "date": "Tue, 26 Aug 2025 21:46:03 GMT"} })
9032025-08-26T21:46:03.325ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1
9042025-08-26T21:46:03.325ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
9052025-08-26T21:46:03.326ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "98370885-45c1-4fb1-baee-26e4abe6f67a", "content-length": "166", "date": "Tue, 26 Aug 2025 21:46:03 GMT"} })
9062025-08-26T21:46:03.327ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
9072025-08-26T21:46:03.327ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
9082025-08-26T21:46:03.327ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa
9092025-08-26T21:46:03.327ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b16f2a9b-96df-4498-a553-dacea9f72223", "content-length": "702", "date": "Tue, 26 Aug 2025 21:46:03 GMT"} })
9102025-08-26T21:46:03.329ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
9112025-08-26T21:46:03.329ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "61630ec6-b7d2-43a9-8d3b-9b6870e5788c", "content-length": "727", "date": "Tue, 26 Aug 2025 21:46:03 GMT"} })
9122025-08-26T21:46:03.332ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
9132025-08-26T21:46:03.332ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a4066dfc-3d74-4dbb-a33e-894473c6695c", "content-length": "729", "date": "Tue, 26 Aug 2025 21:46:03 GMT"} })
9142025-08-26T21:46:03.332ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
9152025-08-26T21:46:03.332ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f27cec77-fd33-4c04-a340-db3dcdd7f36f", "content-length": "728", "date": "Tue, 26 Aug 2025 21:46:03 GMT"} })
9162025-08-26T21:46:03.332ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0
9172025-08-26T21:46:03.332ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "b0cd8d20-8728-421a-8d95-df705df8cd0a", "content-length": "722", "date": "Tue, 26 Aug 2025 21:46:03 GMT"} })
9182025-08-26T21:46:03.333ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/active-slot
9192025-08-26T21:46:03.333ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/active-slot", status: 200, headers: {"content-type": "application/json", "x-request-id": "67ebaf16-6e0d-458f-ad2e-9e4f9273453b", "content-length": "10", "date": "Tue, 26 Aug 2025 21:46:03 GMT"} })
9202025-08-26T21:46:03.333ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/0
9212025-08-26T21:46:03.333ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "596ad889-9cf3-4672-b64b-fb90af1c9e94", "content-length": "144", "date": "Tue, 26 Aug 2025 21:46:03 GMT"} })
9222025-08-26T21:46:03.333ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/1
9232025-08-26T21:46:03.333ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "a843b0f5-2cd2-4168-a0ff-94efa58f75de", "content-length": "147", "date": "Tue, 26 Aug 2025 21:46:03 GMT"} })
9242025-08-26T21:46:03.333ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0
9252025-08-26T21:46:03.334ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "d9f9235c-53c6-4643-8ba9-160fbb7cbea6", "content-length": "109", "date": "Tue, 26 Aug 2025 21:46:03 GMT"} })
9262025-08-26T21:46:03.334ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1
9272025-08-26T21:46:03.334ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "7c984e6f-4348-411b-8ef5-18ce5a60b821", "content-length": "109", "date": "Tue, 26 Aug 2025 21:46:03 GMT"} })
9282025-08-26T21:46:03.334ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0
9292025-08-26T21:46:03.335ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "0a370854-7ef9-446d-9e19-e5d882a6ec1e", "content-length": "163", "date": "Tue, 26 Aug 2025 21:46:03 GMT"} })
9302025-08-26T21:46:03.335ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1
9312025-08-26T21:46:03.335ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "8f52f933-c86a-42a5-8d80-a428cd68a1b7", "content-length": "163", "date": "Tue, 26 Aug 2025 21:46:03 GMT"} })
9322025-08-26T21:46:03.335ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0
9332025-08-26T21:46:03.336ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "d3753add-238c-411a-8743-ffca25030afa", "content-length": "166", "date": "Tue, 26 Aug 2025 21:46:03 GMT"} })
9342025-08-26T21:46:03.336ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1
9352025-08-26T21:46:03.336ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "2ea8b2b4-83e7-43b4-bea9-cbb9e29add17", "content-length": "166", "date": "Tue, 26 Aug 2025 21:46:03 GMT"} })
9362025-08-26T21:46:03.336ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa
9372025-08-26T21:46:03.336ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "666698fe-242f-45db-b72a-8447876ce7b0", "content-length": "702", "date": "Tue, 26 Aug 2025 21:46:03 GMT"} })
9382025-08-26T21:46:03.336ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
9392025-08-26T21:46:03.337ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "503b9e66-8e76-44bc-8997-34e38cd28d62", "content-length": "727", "date": "Tue, 26 Aug 2025 21:46:03 GMT"} })
9402025-08-26T21:46:03.337ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
9412025-08-26T21:46:03.337ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "19db7dd0-7aba-4c00-9e1d-76b05758fdaa", "content-length": "729", "date": "Tue, 26 Aug 2025 21:46:03 GMT"} })
9422025-08-26T21:46:03.337ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
9432025-08-26T21:46:03.337ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "7d4b2f86-72e0-4f83-9da4-4af18a9a812e", "content-length": "728", "date": "Tue, 26 Aug 2025 21:46:03 GMT"} })
9442025-08-26T21:46:03.337ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
9452025-08-26T21:46:03.337ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
9462025-08-26T21:46:03.339ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9472025-08-26T21:46:03.341ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
9482025-08-26T21:46:03.341ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
9492025-08-26T21:46:03.341ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
9502025-08-26T21:46:03.343ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9512025-08-26T21:46:03.346ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
9522025-08-26T21:46:03.346ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
9532025-08-26T21:46:03.346ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
9542025-08-26T21:46:03.372ZDEBGnexus (service-firewall-rule-progator): resolved sleds for vpc oxide-services background_task = service_firewall_rule_propagation vpc_to_sled = []
9552025-08-26T21:46:03.372ZDEBGnexus (service-firewall-rule-progator): sending firewall rules to sled agents background_task = service_firewall_rule_propagation
9562025-08-26T21:46:03.372ZINFOnexus (service-firewall-rule-progator): updated firewall rules on 0 sleds background_task = service_firewall_rule_propagation file = nexus/networking/src/firewall_rules.rs:491
9572025-08-26T21:46:03.372ZDEBGnexus (service-firewall-rule-progator): successfully propagated service firewall rules background_task = service_firewall_rule_propagation elapsed = 512.79804ms
9582025-08-26T21:46:03.372ZDEBGnexus (BackgroundTasks): activation complete background_task = service_firewall_rule_propagation elapsed = 512.884701ms iteration = 1 status = {}
9592025-08-26T21:46:03.376ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
9602025-08-26T21:46:03.376ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
9612025-08-26T21:46:03.378ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9622025-08-26T21:46:03.383ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = VpcRouter { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000001, lookup_type: ById(001de000-074c-4000-8000-000000000001) } result = Ok(())
9632025-08-26T21:46:03.383ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
9642025-08-26T21:46:03.383ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
9652025-08-26T21:46:03.384ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
9662025-08-26T21:46:03.385ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
9672025-08-26T21:46:03.386ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
9682025-08-26T21:46:03.386ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
9692025-08-26T21:46:03.388ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9702025-08-26T21:46:03.393ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = VpcRouter { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000001, lookup_type: ById(001de000-074c-4000-8000-000000000001) } result = Ok(())
9712025-08-26T21:46:03.393ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
9722025-08-26T21:46:03.393ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
9732025-08-26T21:46:03.397ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
9742025-08-26T21:46:03.397ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
9752025-08-26T21:46:03.399ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
9762025-08-26T21:46:03.399ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
9772025-08-26T21:46:03.401ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
9782025-08-26T21:46:03.401ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
9792025-08-26T21:46:03.403ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
9802025-08-26T21:46:03.403ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
9812025-08-26T21:46:03.403ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
9822025-08-26T21:46:03.403ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "76994048-cdde-4c6c-a780-4dd4250d7f71", "content-length": "691", "date": "Tue, 26 Aug 2025 21:46:03 GMT"} })
9832025-08-26T21:46:03.405ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
9842025-08-26T21:46:03.405ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
9852025-08-26T21:46:03.405ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
9862025-08-26T21:46:03.405ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
9872025-08-26T21:46:03.405ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
9882025-08-26T21:46:03.407ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 548.262589ms iteration = 1 status = {}
9892025-08-26T21:46:03.453ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
9902025-08-26T21:46:03.454ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
9912025-08-26T21:46:03.455ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
9922025-08-26T21:46:03.455ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
9932025-08-26T21:46:03.457ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9942025-08-26T21:46:03.461ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ById(001de000-c470-4000-8000-000000000002) } result = Ok(())
9952025-08-26T21:46:03.461ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
9962025-08-26T21:46:03.461ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
9972025-08-26T21:46:03.463ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
9982025-08-26T21:46:03.463ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
9992025-08-26T21:46:03.465ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
10002025-08-26T21:46:03.465ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
10012025-08-26T21:46:03.467ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
10022025-08-26T21:46:03.467ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
10032025-08-26T21:46:03.469ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
10042025-08-26T21:46:03.470ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
10052025-08-26T21:46:03.471ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
10062025-08-26T21:46:03.471ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
10072025-08-26T21:46:03.473ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
10082025-08-26T21:46:03.473ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
10092025-08-26T21:46:03.475ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10102025-08-26T21:46:03.477ZDEBGnexus (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1]:12345
10112025-08-26T21:46:03.477ZDEBGnexus (BackgroundTasks): client request SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/inventory
10122025-08-26T21:46:03.482ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }, key: 001de000-c470-4000-8000-000000000003, lookup_type: ById(001de000-c470-4000-8000-000000000003) } result = Ok(())
10132025-08-26T21:46:03.482ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
10142025-08-26T21:46:03.482ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
10152025-08-26T21:46:03.485ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
10162025-08-26T21:46:03.485ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
10172025-08-26T21:46:03.487ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
10182025-08-26T21:46:03.488ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
10192025-08-26T21:46:03.490ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
10202025-08-26T21:46:03.490ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
10212025-08-26T21:46:03.492ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
10222025-08-26T21:46:03.492ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
10232025-08-26T21:46:03.495ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
10242025-08-26T21:46:03.495ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
10252025-08-26T21:46:03.497ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
10262025-08-26T21:46:03.497ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
10272025-08-26T21:46:03.499ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10282025-08-26T21:46:03.503ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ById(001de000-c470-4000-8000-000000000001) } result = Ok(())
10292025-08-26T21:46:03.503ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
10302025-08-26T21:46:03.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
10312025-08-26T21:46:03.505ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
10322025-08-26T21:46:03.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
10332025-08-26T21:46:03.507ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
10342025-08-26T21:46:03.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
10352025-08-26T21:46:03.512ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
10362025-08-26T21:46:03.512ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
10372025-08-26T21:46:03.515ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
10382025-08-26T21:46:03.515ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
10392025-08-26T21:46:03.516ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
10402025-08-26T21:46:03.516ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
10412025-08-26T21:46:03.518ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
10422025-08-26T21:46:03.518ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
10432025-08-26T21:46:03.520ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10442025-08-26T21:46:03.524ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }, key: 001de000-c470-4000-8000-000000000003, lookup_type: ByName("boundary-ntp") } result = Ok(())
10452025-08-26T21:46:03.524ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
10462025-08-26T21:46:03.524ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
10472025-08-26T21:46:03.526ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
10482025-08-26T21:46:03.526ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
10492025-08-26T21:46:03.527ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
10502025-08-26T21:46:03.528ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
10512025-08-26T21:46:03.529ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
10522025-08-26T21:46:03.529ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
10532025-08-26T21:46:03.532ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
10542025-08-26T21:46:03.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
10552025-08-26T21:46:03.533ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
10562025-08-26T21:46:03.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
10572025-08-26T21:46:03.535ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
10582025-08-26T21:46:03.535ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
10592025-08-26T21:46:03.537ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10602025-08-26T21:46:03.541ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
10612025-08-26T21:46:03.541ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
10622025-08-26T21:46:03.541ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
10632025-08-26T21:46:03.542ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
10642025-08-26T21:46:03.543ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
10652025-08-26T21:46:03.544ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
10662025-08-26T21:46:03.544ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
10672025-08-26T21:46:03.546ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
10682025-08-26T21:46:03.546ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
10692025-08-26T21:46:03.548ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
10702025-08-26T21:46:03.549ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
10712025-08-26T21:46:03.550ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
10722025-08-26T21:46:03.550ZDEBGnexus (BackgroundTasks): client response SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "13d356eb-fc2d-40d1-8aa9-968c0fcaf480", "content-length": "88481", "date": "Tue, 26 Aug 2025 21:46:03 GMT"} })
10732025-08-26T21:46:03.550ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
10742025-08-26T21:46:03.551ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
10752025-08-26T21:46:03.552ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
10762025-08-26T21:46:03.553ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10772025-08-26T21:46:03.556ZDEBGnexus (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
10782025-08-26T21:46:03.556ZDEBGnexus (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
10792025-08-26T21:46:03.556ZDEBGnexus (BackgroundTasks): begin collection from CockroachDB nodes background_task = inventory_collection
10802025-08-26T21:46:03.556ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/node/id
10812025-08-26T21:46:03.556ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/node/id
10822025-08-26T21:46:03.556ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/node/id
10832025-08-26T21:46:03.556ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/node/id
10842025-08-26T21:46:03.556ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/node/id
10852025-08-26T21:46:03.557ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "646f7125-7f37-4a35-a379-76f2bf0b190b", "content-length": "64", "date": "Tue, 26 Aug 2025 21:46:03 GMT"} })
10862025-08-26T21:46:03.557ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/proxy/status/vars
10872025-08-26T21:46:03.557ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "4c90c489-db6d-43e9-9908-d4ddee110d85", "content-length": "64", "date": "Tue, 26 Aug 2025 21:46:03 GMT"} })
10882025-08-26T21:46:03.557ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/proxy/status/vars
10892025-08-26T21:46:03.557ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "f7e6963a-3749-479b-ba57-788e52bb726f", "content-length": "64", "date": "Tue, 26 Aug 2025 21:46:03 GMT"} })
10902025-08-26T21:46:03.557ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/proxy/status/vars
10912025-08-26T21:46:03.558ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
10922025-08-26T21:46:03.558ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
10932025-08-26T21:46:03.558ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
10942025-08-26T21:46:03.559ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "f107cc66-1e1b-4e13-a736-f1989b08748c", "content-length": "64", "date": "Tue, 26 Aug 2025 21:46:03 GMT"} })
10952025-08-26T21:46:03.559ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/proxy/status/vars
10962025-08-26T21:46:03.559ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
10972025-08-26T21:46:03.560ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
10982025-08-26T21:46:03.561ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "4e258a36-72f6-43e0-a865-3058d3deb31d", "content-length": "64", "date": "Tue, 26 Aug 2025 21:46:03 GMT"} })
10992025-08-26T21:46:03.561ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/proxy/status/vars
11002025-08-26T21:46:03.561ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
11012025-08-26T21:46:03.561ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
11022025-08-26T21:46:03.563ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
11032025-08-26T21:46:03.563ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
11042025-08-26T21:46:03.565ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
11052025-08-26T21:46:03.565ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
11062025-08-26T21:46:03.567ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
11072025-08-26T21:46:03.567ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
11082025-08-26T21:46:03.568ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
11092025-08-26T21:46:03.569ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
11102025-08-26T21:46:03.570ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11112025-08-26T21:46:03.574ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = InternetGateway { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000002, lookup_type: ByName("default") } result = Ok(())
11122025-08-26T21:46:03.575ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
11132025-08-26T21:46:03.577ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "f0f51942-f369-437d-af50-f71899ea6c9e", "date": "Tue, 26 Aug 2025 21:46:03 GMT"} })
11142025-08-26T21:46:03.578ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 719.385107ms iteration = 1 status = {}
11152025-08-26T21:46:03.757ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "59c7d0cc-1bb9-4243-a1d2-f201979c5fd8", "content-length": "358327", "date": "Tue, 26 Aug 2025 21:46:03 GMT"} })
11162025-08-26T21:46:03.758ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "b7d2bede-9a72-40a0-9173-101088610ce1", "content-length": "339769", "date": "Tue, 26 Aug 2025 21:46:03 GMT"} })
11172025-08-26T21:46:03.793ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "ffde4e6e-f735-4dfa-b4b6-2e74de92739f", "content-length": "336578", "date": "Tue, 26 Aug 2025 21:46:03 GMT"} })
11182025-08-26T21:46:03.899ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "e9f0a629-147a-442f-8be4-451d6c0e91b3", "content-length": "360637", "date": "Tue, 26 Aug 2025 21:46:03 GMT"} })
11192025-08-26T21:46:03.946ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "fbad7826-07f6-4096-a19c-118147afdef7", "content-length": "376915", "date": "Tue, 26 Aug 2025 21:46:03 GMT"} })
11202025-08-26T21:46:03.948ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 3
11212025-08-26T21:46:03.948ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 4
11222025-08-26T21:46:03.948ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 2
11232025-08-26T21:46:03.948ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 5
11242025-08-26T21:46:03.948ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 1
11252025-08-26T21:46:03.981ZDEBGnexus (BackgroundTasks): begin collection from NTP admin (timesync) background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1b]:10123 zone_id = e17ff130-2852-42b2-9cfd-58a3a8b33eb6 (service)
11262025-08-26T21:46:03.981ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None method = GET ntp_admin_url = http://[fd00:1122:3344:101::1b]:10123 uri = http://[fd00:1122:3344:101::1b]:10123/timesync
11272025-08-26T21:46:04.104ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection ntp_admin_url = http://[fd00:1122:3344:101::1b]:10123 result = Ok(Response { url: "http://[fd00:1122:3344:101::1b]:10123/timesync", status: 200, headers: {"content-type": "application/json", "x-request-id": "2ef7faba-4d9f-4f38-a4c0-207f7ad65d52", "content-length": "124", "date": "Tue, 26 Aug 2025 21:46:04 GMT"} })
11282025-08-26T21:46:04.106ZDEBGnexus (BackgroundTasks): begin collection from internal DNS servers background_task = inventory_collection
11292025-08-26T21:46:04.204ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 852fd1a3-de9a-47e8-a667-5e8dab880ad3 (service)
11302025-08-26T21:46:04.204ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:2::1]:5353 method = GET uri = http://[fd00:1122:3344:2::1]:5353/config
11312025-08-26T21:46:04.274ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:2::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "089e5afa-de14-460f-9776-1c0a319e41dc", "content-length": "9110", "date": "Tue, 26 Aug 2025 21:46:04 GMT"} })
11322025-08-26T21:46:04.274ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 852fd1a3-de9a-47e8-a667-5e8dab880ad3 (service)
11332025-08-26T21:46:04.275ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = c68f1501-294c-4874-92ca-8fc0546a5870 (service)
11342025-08-26T21:46:04.275ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:1::1]:5353 method = GET uri = http://[fd00:1122:3344:1::1]:5353/config
11352025-08-26T21:46:04.336ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:1::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "5032d696-636d-4ea4-abfd-0a4f3d526b58", "content-length": "9110", "date": "Tue, 26 Aug 2025 21:46:04 GMT"} })
11362025-08-26T21:46:04.336ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = c68f1501-294c-4874-92ca-8fc0546a5870 (service)
11372025-08-26T21:46:04.337ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = d741a72f-8745-4723-9e80-e1e72987266b (service)
11382025-08-26T21:46:04.337ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:3::1]:5353 method = GET uri = http://[fd00:1122:3344:3::1]:5353/config
11392025-08-26T21:46:04.402ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:3::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "ef7172be-dbdd-4338-9a87-9674b4332e9e", "content-length": "9110", "date": "Tue, 26 Aug 2025 21:46:04 GMT"} })
11402025-08-26T21:46:04.403ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = d741a72f-8745-4723-9e80-e1e72987266b (service)
11412025-08-26T21:46:04.404ZDEBGnexus (BackgroundTasks): finished collection from internal DNS servers background_task = inventory_collection
11422025-08-26T21:46:04.404ZDEBGnexus (BackgroundTasks): finished collection background_task = inventory_collection
11432025-08-26T21:46:04.406ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
11442025-08-26T21:46:04.406ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
11452025-08-26T21:46:04.408ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11462025-08-26T21:46:04.410ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
11472025-08-26T21:46:04.410ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
11482025-08-26T21:46:04.411ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
11492025-08-26T21:46:05.040ZINFOnexus (BackgroundTasks): inserted inventory collection background_task = inventory_collection collection_id = 98fcdfec-5acc-4b21-be95-18fc820aa3d1 file = nexus/db-queries/src/db/datastore/inventory.rs:1614
11502025-08-26T21:46:05.040ZDEBGnexus (BackgroundTasks): inventory collection complete background_task = inventory_collection collection_id = 98fcdfec-5acc-4b21-be95-18fc820aa3d1 time_started = 2025-08-26 21:46:03.308861 UTC
11512025-08-26T21:46:05.040ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 2 reason = Dependency
11522025-08-26T21:46:05.040ZDEBGnexus (BackgroundTasks): blueprint planning disabled, doing nothing background_task = blueprint_planner
11532025-08-26T21:46:05.040ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_rendezvous iteration = 2 reason = Dependency
11542025-08-26T21:46:05.040ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 9.061\u{b5}s iteration = 2 status = "Disabled"
11552025-08-26T21:46:05.040ZWARNnexus (BackgroundTasks): Blueprint rendezvous: skipped background_task = blueprint_rendezvous file = nexus/src/app/background/tasks/blueprint_rendezvous.rs:52 reason = no blueprint
11562025-08-26T21:46:05.040ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 2 reason = Dependency
11572025-08-26T21:46:05.040ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_rendezvous elapsed = 16.221\u{b5}s iteration = 2 status = {"error":"no blueprint"}
11582025-08-26T21:46:05.040ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
11592025-08-26T21:46:05.040ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 2.200365141s iteration = 1 status = {"collection_id":"98fcdfec-5acc-4b21-be95-18fc820aa3d1","time_done":"2025-08-26 21:46:04.404646 UTC","time_started":"2025-08-26 21:46:03.308861 UTC"}
11602025-08-26T21:46:05.040ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
11612025-08-26T21:46:05.042ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11622025-08-26T21:46:05.045ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
11632025-08-26T21:46:05.045ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
11642025-08-26T21:46:05.045ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
11652025-08-26T21:46:05.047ZINFOnexus (BackgroundTasks): Physical Disk Adoption: Rack not yet initialized background_task = physical_disk_adoption file = nexus/src/app/background/tasks/physical_disk_adoption.rs:81 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
11662025-08-26T21:46:05.047ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 7.339985ms iteration = 2 status = {"error":"rack not yet initialized: e37a46f2-f789-4ca4-9841-f8c6dd705964"}
11672025-08-26T21:46:05.145ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
11682025-08-26T21:46:05.145ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
11692025-08-26T21:46:05.148ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
11702025-08-26T21:46:05.148ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
11712025-08-26T21:46:05.150ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11722025-08-26T21:46:05.153ZDEBGnexus (ServiceBalancer): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Rack { parent: Fleet, key: e37a46f2-f789-4ca4-9841-f8c6dd705964, lookup_type: ById(e37a46f2-f789-4ca4-9841-f8c6dd705964) } result = Ok(())
11732025-08-26T21:46:05.153ZINFOnexus: Still waiting for rack initialization: Rack { identity: RackIdentity { id: e37a46f2-f789-4ca4-9841-f8c6dd705964, time_created: 2025-08-26T21:46:02.666890Z, time_modified: 2025-08-26T21:46:02.666890Z }, initialized: false, tuf_base_url: None, rack_subnet: Some(V6(Ipv6Network { addr: fd00:1122:3344:100::, prefix: 56 })) } file = nexus/src/app/rack.rs:775
11742025-08-26T21:46:07.154ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
11752025-08-26T21:46:07.154ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
11762025-08-26T21:46:07.157ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
11772025-08-26T21:46:07.157ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
11782025-08-26T21:46:07.159ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11792025-08-26T21:46:07.162ZDEBGnexus (ServiceBalancer): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Rack { parent: Fleet, key: e37a46f2-f789-4ca4-9841-f8c6dd705964, lookup_type: ById(e37a46f2-f789-4ca4-9841-f8c6dd705964) } result = Ok(())
11802025-08-26T21:46:07.162ZINFOnexus: Rack initialized file = nexus/src/app/rack.rs:772
11812025-08-26T21:46:07.162ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
11822025-08-26T21:46:07.162ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
11832025-08-26T21:46:07.164ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
11842025-08-26T21:46:07.164ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
11852025-08-26T21:46:07.168ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
11862025-08-26T21:46:07.168ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
11872025-08-26T21:46:07.170ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
11882025-08-26T21:46:07.170ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
11892025-08-26T21:46:07.173ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
11902025-08-26T21:46:07.173ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
11912025-08-26T21:46:07.175ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
11922025-08-26T21:46:07.175ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
11932025-08-26T21:46:07.177ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11942025-08-26T21:46:07.180ZDEBGnexus (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
11952025-08-26T21:46:07.180ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
11962025-08-26T21:46:07.180ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
11972025-08-26T21:46:07.182ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
11982025-08-26T21:46:07.182ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
11992025-08-26T21:46:07.184ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
12002025-08-26T21:46:07.184ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
12012025-08-26T21:46:07.185ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12022025-08-26T21:46:07.188ZDEBGnexus (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
12032025-08-26T21:46:07.188ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
12042025-08-26T21:46:07.188ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
12052025-08-26T21:46:07.191ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
12062025-08-26T21:46:07.191ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
12072025-08-26T21:46:07.193ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
12082025-08-26T21:46:07.193ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
12092025-08-26T21:46:07.194ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
12102025-08-26T21:46:07.194ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
12112025-08-26T21:46:07.196ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
12122025-08-26T21:46:07.196ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
12132025-08-26T21:46:07.197ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
12142025-08-26T21:46:07.197ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
12152025-08-26T21:46:07.199ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
12162025-08-26T21:46:07.199ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
12172025-08-26T21:46:07.200ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12182025-08-26T21:46:07.203ZDEBGnexus (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
12192025-08-26T21:46:07.203ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
12202025-08-26T21:46:07.204ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
12212025-08-26T21:46:07.205ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
12222025-08-26T21:46:07.205ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
12232025-08-26T21:46:07.207ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
12242025-08-26T21:46:07.207ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
12252025-08-26T21:46:07.209ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
12262025-08-26T21:46:07.209ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
12272025-08-26T21:46:07.214ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
12282025-08-26T21:46:07.214ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
12292025-08-26T21:46:07.216ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
12302025-08-26T21:46:07.216ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
12312025-08-26T21:46:07.217ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
12322025-08-26T21:46:07.217ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
12332025-08-26T21:46:07.219ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12342025-08-26T21:46:07.223ZDEBGnexus (InternalApi): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
12352025-08-26T21:46:07.223ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
12362025-08-26T21:46:07.223ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
12372025-08-26T21:46:07.224ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
12382025-08-26T21:46:07.224ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
12392025-08-26T21:46:07.226ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
12402025-08-26T21:46:07.226ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
12412025-08-26T21:46:07.227ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12422025-08-26T21:46:07.231ZDEBGnexus (InternalApi): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
12432025-08-26T21:46:07.231ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
12442025-08-26T21:46:07.231ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
12452025-08-26T21:46:07.235ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
12462025-08-26T21:46:07.235ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
12472025-08-26T21:46:07.236ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
12482025-08-26T21:46:07.237ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
12492025-08-26T21:46:07.238ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
12502025-08-26T21:46:07.238ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
12512025-08-26T21:46:07.240ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
12522025-08-26T21:46:07.240ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
12532025-08-26T21:46:07.242ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
12542025-08-26T21:46:07.243ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
12552025-08-26T21:46:07.244ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
12562025-08-26T21:46:07.244ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
12572025-08-26T21:46:07.245ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
12582025-08-26T21:46:07.246ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
12592025-08-26T21:46:07.247ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12602025-08-26T21:46:07.251ZDEBGnexus (InternalApi): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
12612025-08-26T21:46:07.251ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
12622025-08-26T21:46:07.251ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
12632025-08-26T21:46:07.253ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
12642025-08-26T21:46:07.253ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
12652025-08-26T21:46:07.254ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
12662025-08-26T21:46:07.254ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
12672025-08-26T21:46:07.255ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12682025-08-26T21:46:07.259ZDEBGnexus (InternalApi): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
12692025-08-26T21:46:07.259ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
12702025-08-26T21:46:07.259ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
12712025-08-26T21:46:07.266ZDEBGnexus (InternalApi): resolved names for firewall rules instance_interfaces = {} subnet_interfaces = {Name("nexus"): [NetworkInterface { id: 0802405a-a5f2-402c-94da-ccafb58bccb5, kind: Service { id: 4996dcc4-c530-4f64-b407-fa372462bd9e }, name: Name("nexus-4996dcc4-c530-4f64-b407-fa372462bd9e"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 186, 56])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 1acb6bfd-558b-4271-bbf5-58e0d089b710, kind: Service { id: 0205eaf0-3fc6-4ad5-b09b-732af233e9c1 }, name: Name("nexus-0205eaf0-3fc6-4ad5-b09b-732af233e9c1"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 243, 142])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: cbb99fa2-edc6-497d-9c47-8dd2642f8945, kind: Service { id: 5d0959e5-47a6-4bf3-a52b-57950b9d900d }, name: Name("nexus-5d0959e5-47a6-4bf3-a52b-57950b9d900d"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 162, 80])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], Name("external-dns"): [NetworkInterface { id: 0106180c-e6f8-4d65-969e-4fbbddf596a1, kind: Service { id: 60abf731-31e1-4111-a3e2-65da89eaed87 }, name: Name("external-dns-60abf731-31e1-4111-a3e2-65da89eaed87"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 156, 81])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: b69ae953-8f67-4cb3-9521-db9e707a6f2d, kind: Service { id: 84c849eb-0949-41d9-ad02-9a5509144564 }, name: Name("external-dns-84c849eb-0949-41d9-ad02-9a5509144564"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 240, 136])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }]} subnet_networks = {Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })], Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })]} vpc_interfaces = {}
12722025-08-26T21:46:07.266ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
12732025-08-26T21:46:07.266ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
12742025-08-26T21:46:07.267ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12752025-08-26T21:46:07.269ZDEBGnexus (InternalApi): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet result = Ok(())
12762025-08-26T21:46:07.269ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
12772025-08-26T21:46:07.270ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
12782025-08-26T21:46:07.275ZDEBGnexus (InternalApi): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified.
12792025-08-26T21:46:07.277ZDEBGnexus (InternalApi): resolved firewall rules for sled agents sled_agent_rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 0106180c-e6f8-4d65-969e-4fbbddf596a1, kind: Service { id: 60abf731-31e1-4111-a3e2-65da89eaed87 }, name: Name("external-dns-60abf731-31e1-4111-a3e2-65da89eaed87"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 156, 81])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: b69ae953-8f67-4cb3-9521-db9e707a6f2d, kind: Service { id: 84c849eb-0949-41d9-ad02-9a5509144564 }, name: Name("external-dns-84c849eb-0949-41d9-ad02-9a5509144564"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 240, 136])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 0802405a-a5f2-402c-94da-ccafb58bccb5, kind: Service { id: 4996dcc4-c530-4f64-b407-fa372462bd9e }, name: Name("nexus-4996dcc4-c530-4f64-b407-fa372462bd9e"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 186, 56])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 1acb6bfd-558b-4271-bbf5-58e0d089b710, kind: Service { id: 0205eaf0-3fc6-4ad5-b09b-732af233e9c1 }, name: Name("nexus-0205eaf0-3fc6-4ad5-b09b-732af233e9c1"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 243, 142])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: cbb99fa2-edc6-497d-9c47-8dd2642f8945, kind: Service { id: 5d0959e5-47a6-4bf3-a52b-57950b9d900d }, name: Name("nexus-5d0959e5-47a6-4bf3-a52b-57950b9d900d"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 162, 80])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 0106180c-e6f8-4d65-969e-4fbbddf596a1, kind: Service { id: 60abf731-31e1-4111-a3e2-65da89eaed87 }, name: Name("external-dns-60abf731-31e1-4111-a3e2-65da89eaed87"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 156, 81])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: b69ae953-8f67-4cb3-9521-db9e707a6f2d, kind: Service { id: 84c849eb-0949-41d9-ad02-9a5509144564 }, name: Name("external-dns-84c849eb-0949-41d9-ad02-9a5509144564"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 240, 136])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: None, filter_protocols: Some([Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 3, code: Some(IcmpParamRange { first: 3, last: 4 }) })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 5, code: None })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 11, code: None }))]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 0802405a-a5f2-402c-94da-ccafb58bccb5, kind: Service { id: 4996dcc4-c530-4f64-b407-fa372462bd9e }, name: Name("nexus-4996dcc4-c530-4f64-b407-fa372462bd9e"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 186, 56])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 1acb6bfd-558b-4271-bbf5-58e0d089b710, kind: Service { id: 0205eaf0-3fc6-4ad5-b09b-732af233e9c1 }, name: Name("nexus-0205eaf0-3fc6-4ad5-b09b-732af233e9c1"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 243, 142])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: cbb99fa2-edc6-497d-9c47-8dd2642f8945, kind: Service { id: 5d0959e5-47a6-4bf3-a52b-57950b9d900d }, name: Name("nexus-5d0959e5-47a6-4bf3-a52b-57950b9d900d"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 162, 80])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }]
12802025-08-26T21:46:07.277ZDEBGnexus (InternalApi): resolved 3 rules for sleds
12812025-08-26T21:46:07.345ZDEBGnexus (InternalApi): resolved sleds for vpc oxide-services vpc_to_sled = [Sled { identity: SledIdentity { id: dd0c655a-95c2-48fc-b4c3-9733ed4b0faa, time_created: 2025-08-26T21:46:02.113579Z, time_modified: 2025-08-26T21:46:02.113579Z }, time_deleted: None, rcgen: Generation(Generation(19)), rack_id: e37a46f2-f789-4ca4-9841-f8c6dd705964, is_scrimlet: true, serial_number: "sock", part_number: "i86pc", revision: SqlU32(0), usable_hardware_threads: SqlU32(48), usable_physical_ram: ByteCount(ByteCount(102952026112)), reservoir_size: ByteCount(ByteCount(56094621696)), ip: fd00:1122:3344:101::1, port: SqlU16(12345), last_used_address: fd00:1122:3344:101::ffff, policy: InService, state: Active, sled_agent_gen: Generation(Generation(1)), repo_depot_port: SqlU16(12348), cpu_family: AmdMilan }]
12822025-08-26T21:46:07.345ZDEBGnexus (InternalApi): sending firewall rules to sled agents
12832025-08-26T21:46:07.381ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
12842025-08-26T21:46:07.381ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
12852025-08-26T21:46:07.385ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
12862025-08-26T21:46:07.385ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
12872025-08-26T21:46:07.386ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12882025-08-26T21:46:07.389ZDEBGnexus (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Sled { parent: Fleet, key: dd0c655a-95c2-48fc-b4c3-9733ed4b0faa, lookup_type: ById(dd0c655a-95c2-48fc-b4c3-9733ed4b0faa) } result = Ok(())
12892025-08-26T21:46:07.389ZDEBGnexus (InternalApi): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules
12902025-08-26T21:46:07.390ZDEBGnexus (InternalApi): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules", status: 204, headers: {"x-request-id": "9e9a1165-a782-48d6-bc53-7e13f4838020", "date": "Tue, 26 Aug 2025 21:46:06 GMT"} })
12912025-08-26T21:46:07.391ZINFOnexus (InternalApi): updated firewall rules on 1 sleds file = nexus/networking/src/firewall_rules.rs:491
12922025-08-26T21:46:07.391ZINFOnexus: plumbed initial IP allowlist file = nexus/src/app/allow_list.rs:161
12932025-08-26T21:46:07.840ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 2 reason = Timeout
12942025-08-26T21:46:07.840ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
12952025-08-26T21:46:07.840ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
12962025-08-26T21:46:07.842ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12972025-08-26T21:46:07.845ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
12982025-08-26T21:46:07.845ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
12992025-08-26T21:46:07.845ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
13002025-08-26T21:46:07.847ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
13012025-08-26T21:46:07.847ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.191402ms iteration = 2 status = {"chicken_switches_updated":false}
13022025-08-26T21:46:12.840ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 3 reason = Timeout
13032025-08-26T21:46:12.840ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
13042025-08-26T21:46:12.840ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
13052025-08-26T21:46:12.842ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13062025-08-26T21:46:12.845ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
13072025-08-26T21:46:12.845ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
13082025-08-26T21:46:12.845ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
13092025-08-26T21:46:12.855ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
13102025-08-26T21:46:12.855ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 15.456569ms iteration = 3 status = {"chicken_switches_updated":false}
13112025-08-26T21:46:12.858ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 2 reason = Timeout
13122025-08-26T21:46:12.858ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
13132025-08-26T21:46:12.858ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 2 reason = Timeout
13142025-08-26T21:46:12.858ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
13152025-08-26T21:46:12.859ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
13162025-08-26T21:46:12.859ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
13172025-08-26T21:46:12.860ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13182025-08-26T21:46:12.863ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
13192025-08-26T21:46:12.863ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
13202025-08-26T21:46:12.863ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
13212025-08-26T21:46:12.866ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
13222025-08-26T21:46:12.866ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
13232025-08-26T21:46:12.868ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13242025-08-26T21:46:12.871ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
13252025-08-26T21:46:12.871ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
13262025-08-26T21:46:12.872ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
13272025-08-26T21:46:12.908ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
13282025-08-26T21:46:12.908ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
13292025-08-26T21:46:12.947ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 88.962906ms iteration = 2 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
13302025-08-26T21:46:13.007ZINFOnexus (BackgroundTasks): found new target blueprint (first find) background_task = blueprint_loader file = nexus/src/app/background/tasks/blueprint_load.rs:88 target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
13312025-08-26T21:46:13.007ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 148.824445ms iteration = 2 status = {"enabled":false,"status":"first target blueprint","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z","time_found":"2025-08-26T21:46:13.007717220Z"}
13322025-08-26T21:46:13.007ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 2 reason = Dependency
13332025-08-26T21:46:13.007ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:91 reason = blueprint disabled target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8
13342025-08-26T21:46:13.007ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 3 reason = Dependency
13352025-08-26T21:46:13.007ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 18.14\u{b5}s iteration = 2 status = {"enabled":false,"target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8"}
13362025-08-26T21:46:13.007ZDEBGnexus (BackgroundTasks): blueprint planning disabled, doing nothing background_task = blueprint_planner
13372025-08-26T21:46:13.007ZDEBGnexus (BackgroundTasks): activating background_task = crdb_node_id_collector iteration = 2 reason = Dependency
13382025-08-26T21:46:13.007ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 17.151\u{b5}s iteration = 3 status = "Disabled"
13392025-08-26T21:46:13.007ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
13402025-08-26T21:46:13.008ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
13412025-08-26T21:46:13.008ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
13422025-08-26T21:46:13.008ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
13432025-08-26T21:46:13.008ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
13442025-08-26T21:46:13.008ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
13452025-08-26T21:46:13.008ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
13462025-08-26T21:46:13.008ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
13472025-08-26T21:46:13.008ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
13482025-08-26T21:46:13.009ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
13492025-08-26T21:46:13.010ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13502025-08-26T21:46:13.012ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
13512025-08-26T21:46:13.013ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
13522025-08-26T21:46:13.013ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
13532025-08-26T21:46:13.015ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13542025-08-26T21:46:13.017ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
13552025-08-26T21:46:13.017ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
13562025-08-26T21:46:13.017ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
13572025-08-26T21:46:13.018ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13582025-08-26T21:46:13.019ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
13592025-08-26T21:46:13.019ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
13602025-08-26T21:46:13.020ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
13612025-08-26T21:46:13.020ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13622025-08-26T21:46:13.022ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
13632025-08-26T21:46:13.022ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
13642025-08-26T21:46:13.022ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
13652025-08-26T21:46:13.022ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13662025-08-26T21:46:13.024ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
13672025-08-26T21:46:13.024ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
13682025-08-26T21:46:13.024ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
13692025-08-26T21:46:13.026ZDEBGnexus (BackgroundTasks): activation complete background_task = crdb_node_id_collector elapsed = 18.469407ms iteration = 2 status = {"nsuccess":5}
13702025-08-26T21:46:17.839ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 4 reason = Timeout
13712025-08-26T21:46:17.839ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
13722025-08-26T21:46:17.840ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
13732025-08-26T21:46:17.841ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13742025-08-26T21:46:17.844ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
13752025-08-26T21:46:17.844ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
13762025-08-26T21:46:17.844ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
13772025-08-26T21:46:17.846ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
13782025-08-26T21:46:17.846ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 6.897586ms iteration = 4 status = {"chicken_switches_updated":false}
13792025-08-26T21:46:22.840ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 5 reason = Timeout
13802025-08-26T21:46:22.840ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
13812025-08-26T21:46:22.840ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
13822025-08-26T21:46:22.842ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13832025-08-26T21:46:22.845ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
13842025-08-26T21:46:22.845ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
13852025-08-26T21:46:22.845ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
13862025-08-26T21:46:22.847ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
13872025-08-26T21:46:22.847ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.641583ms iteration = 5 status = {"chicken_switches_updated":false}
13882025-08-26T21:46:22.859ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 3 reason = Timeout
13892025-08-26T21:46:22.859ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 3 reason = Timeout
13902025-08-26T21:46:22.859ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
13912025-08-26T21:46:22.859ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
13922025-08-26T21:46:22.859ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
13932025-08-26T21:46:22.859ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
13942025-08-26T21:46:22.861ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13952025-08-26T21:46:22.864ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
13962025-08-26T21:46:22.864ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
13972025-08-26T21:46:22.864ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
13982025-08-26T21:46:22.864ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
13992025-08-26T21:46:22.864ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
14002025-08-26T21:46:22.866ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
14012025-08-26T21:46:22.866ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
14022025-08-26T21:46:22.868ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14032025-08-26T21:46:22.871ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
14042025-08-26T21:46:22.871ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
14052025-08-26T21:46:22.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
14062025-08-26T21:46:22.889ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
14072025-08-26T21:46:22.889ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 30.309173ms iteration = 3 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
14082025-08-26T21:46:22.900ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 41.306512ms iteration = 3 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
14092025-08-26T21:46:27.840ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 6 reason = Timeout
14102025-08-26T21:46:27.840ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
14112025-08-26T21:46:27.840ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
14122025-08-26T21:46:27.842ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14132025-08-26T21:46:27.845ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
14142025-08-26T21:46:27.845ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
14152025-08-26T21:46:27.845ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
14162025-08-26T21:46:27.847ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
14172025-08-26T21:46:27.847ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.330775ms iteration = 6 status = {"chicken_switches_updated":false}
14182025-08-26T21:46:30.343ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:1025 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::d]:33372
14192025-08-26T21:46:30.367ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:33372 req_id = 793ae606-312e-43de-8be7-e8ab5dd95199 roles = RoleSet { roles: {} } uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?sort_by=id_ascending
14202025-08-26T21:46:30.368ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:33372 req_id = 793ae606-312e-43de-8be7-e8ab5dd95199 resource = Database result = Ok(()) uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?sort_by=id_ascending
14212025-08-26T21:46:30.392ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 47983 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:33372 req_id = 793ae606-312e-43de-8be7-e8ab5dd95199 response_code = 200 uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?sort_by=id_ascending
14222025-08-26T21:46:30.392ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:33372 req_id = 39cfba4f-52bb-4ae9-90d0-42c4195faccc roles = RoleSet { roles: {} } uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkMGJkNjIxMC00NzEyLTQ1NTMtOTg1NC05NjdkODIzNGU4MTUifX0%3D
14232025-08-26T21:46:30.392ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:33372 req_id = 39cfba4f-52bb-4ae9-90d0-42c4195faccc resource = Database result = Ok(()) uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkMGJkNjIxMC00NzEyLTQ1NTMtOTg1NC05NjdkODIzNGU4MTUifX0%3D
14242025-08-26T21:46:30.397ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 5282 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:33372 req_id = 39cfba4f-52bb-4ae9-90d0-42c4195faccc response_code = 200 uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkMGJkNjIxMC00NzEyLTQ1NTMtOTg1NC05NjdkODIzNGU4MTUifX0%3D
14252025-08-26T21:46:32.832ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 2 reason = Timeout
14262025-08-26T21:46:32.832ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 2 reason = Timeout
14272025-08-26T21:46:32.832ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 2 reason = Timeout
14282025-08-26T21:46:32.832ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
14292025-08-26T21:46:32.832ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
14302025-08-26T21:46:32.833ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
14312025-08-26T21:46:32.833ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
14322025-08-26T21:46:32.835ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 896, tv_nsec: 695357570 } })
14332025-08-26T21:46:32.836ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 896, tv_nsec: 695502883 } })
14342025-08-26T21:46:32.836ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1205
14352025-08-26T21:46:32.836ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1205
14362025-08-26T21:46:32.845ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 12.619285ms iteration = 2 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
14372025-08-26T21:46:32.845ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 7 reason = Timeout
14382025-08-26T21:46:32.845ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
14392025-08-26T21:46:32.845ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
14402025-08-26T21:46:32.847ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14412025-08-26T21:46:32.850ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
14422025-08-26T21:46:32.850ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
14432025-08-26T21:46:32.850ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
14442025-08-26T21:46:32.851ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
14452025-08-26T21:46:32.852ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 6.69264ms iteration = 7 status = {"chicken_switches_updated":false}
14462025-08-26T21:46:32.859ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 2 reason = Timeout
14472025-08-26T21:46:32.859ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 2 reason = Timeout
14482025-08-26T21:46:32.859ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 2 reason = Timeout
14492025-08-26T21:46:32.859ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
14502025-08-26T21:46:32.859ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
14512025-08-26T21:46:32.859ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
14522025-08-26T21:46:32.859ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 4 reason = Timeout
14532025-08-26T21:46:32.859ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
14542025-08-26T21:46:32.859ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
14552025-08-26T21:46:32.859ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 2 reason = Timeout
14562025-08-26T21:46:32.859ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 4 reason = Timeout
14572025-08-26T21:46:32.859ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
14582025-08-26T21:46:32.859ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
14592025-08-26T21:46:32.859ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 2 reason = Timeout
14602025-08-26T21:46:32.859ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
14612025-08-26T21:46:32.859ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
14622025-08-26T21:46:32.859ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 3 reason = Timeout
14632025-08-26T21:46:32.859ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 2 reason = Timeout
14642025-08-26T21:46:32.859ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
14652025-08-26T21:46:32.859ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
14662025-08-26T21:46:32.859ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 2 reason = Timeout
14672025-08-26T21:46:32.859ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
14682025-08-26T21:46:32.859ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 2 reason = Timeout
14692025-08-26T21:46:32.859ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
14702025-08-26T21:46:32.859ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
14712025-08-26T21:46:32.859ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
14722025-08-26T21:46:32.859ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
14732025-08-26T21:46:32.859ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 2 reason = Timeout
14742025-08-26T21:46:32.859ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 8.55\u{b5}s iteration = 2 status = {"disabled":true,"errors":[],"sps":[]}
14752025-08-26T21:46:32.859ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 2 reason = Timeout
14762025-08-26T21:46:32.859ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 2 reason = Timeout
14772025-08-26T21:46:32.859ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 2 reason = Timeout
14782025-08-26T21:46:32.859ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
14792025-08-26T21:46:32.859ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
14802025-08-26T21:46:32.859ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
14812025-08-26T21:46:32.859ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 2 reason = Timeout
14822025-08-26T21:46:32.859ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
14832025-08-26T21:46:32.859ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 2 reason = Timeout
14842025-08-26T21:46:32.859ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
14852025-08-26T21:46:32.859ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
14862025-08-26T21:46:32.859ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
14872025-08-26T21:46:32.860ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
14882025-08-26T21:46:32.860ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
14892025-08-26T21:46:32.860ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
14902025-08-26T21:46:32.860ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
14912025-08-26T21:46:32.860ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
14922025-08-26T21:46:32.860ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
14932025-08-26T21:46:32.860ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
14942025-08-26T21:46:32.860ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
14952025-08-26T21:46:32.860ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
14962025-08-26T21:46:32.860ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
14972025-08-26T21:46:32.860ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
14982025-08-26T21:46:32.861ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14992025-08-26T21:46:32.861ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
15002025-08-26T21:46:32.861ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15012025-08-26T21:46:32.861ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15022025-08-26T21:46:32.861ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15032025-08-26T21:46:32.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
15042025-08-26T21:46:32.862ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15052025-08-26T21:46:32.862ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15062025-08-26T21:46:32.862ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15072025-08-26T21:46:32.865ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
15082025-08-26T21:46:32.865ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
15092025-08-26T21:46:32.866ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
15102025-08-26T21:46:32.866ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
15112025-08-26T21:46:32.866ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
15122025-08-26T21:46:32.866ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
15132025-08-26T21:46:32.867ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
15142025-08-26T21:46:32.867ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
15152025-08-26T21:46:32.867ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
15162025-08-26T21:46:32.867ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
15172025-08-26T21:46:32.867ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15182025-08-26T21:46:32.867ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
15192025-08-26T21:46:32.867ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
15202025-08-26T21:46:32.867ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
15212025-08-26T21:46:32.867ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
15222025-08-26T21:46:32.868ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
15232025-08-26T21:46:32.868ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
15242025-08-26T21:46:32.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
15252025-08-26T21:46:32.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
15262025-08-26T21:46:32.868ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
15272025-08-26T21:46:32.868ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
15282025-08-26T21:46:32.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
15292025-08-26T21:46:32.870ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
15302025-08-26T21:46:32.870ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
15312025-08-26T21:46:32.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
15322025-08-26T21:46:32.870ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
15332025-08-26T21:46:32.870ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 896, tv_nsec: 730246376 } })
15342025-08-26T21:46:32.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
15352025-08-26T21:46:32.871ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1205 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
15362025-08-26T21:46:32.871ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
15372025-08-26T21:46:32.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
15382025-08-26T21:46:32.873ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15392025-08-26T21:46:32.875ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
15402025-08-26T21:46:32.875ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
15412025-08-26T21:46:32.875ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
15422025-08-26T21:46:32.894ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1213 zone_address = fd00:1122:3344:101::2
15432025-08-26T21:46:32.894ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
15442025-08-26T21:46:32.895ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "3f6a57b1-e3f8-40c5-b8a6-6ee923eb6241", "content-length": "26", "date": "Tue, 26 Aug 2025 21:46:32 GMT"} })
15452025-08-26T21:46:32.895ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1216 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
15462025-08-26T21:46:32.896ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
15472025-08-26T21:46:32.898ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1213 zone_address = fd00:1122:3344:101::2
15482025-08-26T21:46:32.898ZDEBGnexus (MgsClient): client request background_task = nat_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
15492025-08-26T21:46:32.898ZDEBGnexus (MgsClient): client response background_task = nat_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "292ffed9-19c1-466b-b39a-d1a7614a8111", "content-length": "26", "date": "Tue, 26 Aug 2025 21:46:32 GMT"} })
15502025-08-26T21:46:32.898ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1216 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
15512025-08-26T21:46:32.899ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
15522025-08-26T21:46:32.913ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 53.452605ms iteration = 2 status = {"errors":[],"finish_invoked_ok":[]}
15532025-08-26T21:46:32.914ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
15542025-08-26T21:46:32.914ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
15552025-08-26T21:46:32.914ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
15562025-08-26T21:46:32.914ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
15572025-08-26T21:46:32.917ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
15582025-08-26T21:46:32.917ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
15592025-08-26T21:46:32.918ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
15602025-08-26T21:46:32.918ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 58.208912ms iteration = 2 status = {"errors":[],"requests_created_ok":[]}
15612025-08-26T21:46:32.918ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
15622025-08-26T21:46:32.918ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
15632025-08-26T21:46:32.918ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 59.304227ms iteration = 4 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
15642025-08-26T21:46:32.919ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
15652025-08-26T21:46:32.920ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
15662025-08-26T21:46:32.921ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
15672025-08-26T21:46:32.930ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:550 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
15682025-08-26T21:46:32.930ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 70.432007ms iteration = 2 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
15692025-08-26T21:46:32.955ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 96.320711ms iteration = 2 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
15702025-08-26T21:46:32.960ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
15712025-08-26T21:46:32.960ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
15722025-08-26T21:46:32.961ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15732025-08-26T21:46:32.961ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
15742025-08-26T21:46:32.961ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
15752025-08-26T21:46:32.961ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
15762025-08-26T21:46:32.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
15772025-08-26T21:46:32.963ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
15782025-08-26T21:46:32.964ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
15792025-08-26T21:46:32.964ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
15802025-08-26T21:46:32.964ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 104.487565ms iteration = 2 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
15812025-08-26T21:46:32.983ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1213 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 zone_address = fd00:1122:3344:101::2
15822025-08-26T21:46:32.983ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
15832025-08-26T21:46:32.983ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "25e758f2-8921-40ee-b997-243afac974a1", "content-length": "26", "date": "Tue, 26 Aug 2025 21:46:32 GMT"} })
15842025-08-26T21:46:32.983ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1216 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
15852025-08-26T21:46:32.985ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
15862025-08-26T21:46:32.988ZINFOnexus (BackgroundTasks): VPC route manager sled dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
15872025-08-26T21:46:32.988ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
15882025-08-26T21:46:32.989ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "e386cd3d-7a56-4faa-9ce7-d00664125526", "content-length": "372", "date": "Tue, 26 Aug 2025 21:46:32 GMT"} })
15892025-08-26T21:46:32.989ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
15902025-08-26T21:46:32.989ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
15912025-08-26T21:46:32.994ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 134.745958ms iteration = 2 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
15922025-08-26T21:46:32.998ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
15932025-08-26T21:46:32.998ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
15942025-08-26T21:46:32.998ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 139.166277ms iteration = 2 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
15952025-08-26T21:46:33.000ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:407
15962025-08-26T21:46:33.000ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 141.012149ms iteration = 2 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
15972025-08-26T21:46:33.000ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 168.042058ms iteration = 2 status = {}
15982025-08-26T21:46:33.010ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
15992025-08-26T21:46:33.010ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
16002025-08-26T21:46:33.012ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 180.202093ms iteration = 2 status = null
16012025-08-26T21:46:33.051ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
16022025-08-26T21:46:33.051ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
16032025-08-26T21:46:33.051ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "d61a4d94-f28a-40e5-b601-fc4c7359ffef", "date": "Tue, 26 Aug 2025 21:46:32 GMT"} })
16042025-08-26T21:46:33.051ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
16052025-08-26T21:46:33.051ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
16062025-08-26T21:46:33.053ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
16072025-08-26T21:46:33.054ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
16082025-08-26T21:46:33.057ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
16092025-08-26T21:46:33.057ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
16102025-08-26T21:46:33.060ZINFOnexus (BackgroundTasks): VPC route manager sled dd0c655a-95c2-48fc-b4c3-9733ed4b0faa push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
16112025-08-26T21:46:33.060ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
16122025-08-26T21:46:33.061ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "ed1d30c3-0a44-408d-bbec-d3b9a8545d09", "date": "Tue, 26 Aug 2025 21:46:32 GMT"} })
16132025-08-26T21:46:33.062ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 202.946535ms iteration = 2 status = {}
16142025-08-26T21:46:33.062ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 203.000796ms iteration = 3 status = {"physical_disks_added":0}
16152025-08-26T21:46:33.102ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
16162025-08-26T21:46:33.102ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
16172025-08-26T21:46:33.104ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: d24c5c55-3a21-4a91-aeb4-45362e822df6, rack_id: e37a46f2-f789-4ca4-9841-f8c6dd705964, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(88d84f37-a4c8-4abc-af1f-27987b0d39d9) } rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0
16182025-08-26T21:46:33.105ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
16192025-08-26T21:46:33.105ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
16202025-08-26T21:46:33.203ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1832 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
16212025-08-26T21:46:33.203ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1859 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 88d84f37-a4c8-4abc-af1f-27987b0d39d9
16222025-08-26T21:46:33.206ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
16232025-08-26T21:46:33.206ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
16242025-08-26T21:46:33.206ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
16252025-08-26T21:46:33.206ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
16262025-08-26T21:46:33.206ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
16272025-08-26T21:46:33.206ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2006 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
16282025-08-26T21:46:33.207ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:408 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
16292025-08-26T21:46:33.207ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2042 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
16302025-08-26T21:46:33.207ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
16312025-08-26T21:46:33.207ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
16322025-08-26T21:46:33.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
16332025-08-26T21:46:33.223ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0
16342025-08-26T21:46:33.224ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
16352025-08-26T21:46:33.242ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
16362025-08-26T21:46:33.242ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
16372025-08-26T21:46:33.245ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
16382025-08-26T21:46:33.246ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
16392025-08-26T21:46:33.247ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16402025-08-26T21:46:33.250ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: dd0c655a-95c2-48fc-b4c3-9733ed4b0faa, lookup_type: ById(dd0c655a-95c2-48fc-b4c3-9733ed4b0faa) } result = Ok(())
16412025-08-26T21:46:33.250ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
16422025-08-26T21:46:33.250ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
16432025-08-26T21:46:33.252ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 393.04133ms iteration = 2 status = 0
16442025-08-26T21:46:33.329ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "c9ecff29-4b6c-41d4-8490-ba93cd39c715", "date": "Tue, 26 Aug 2025 21:46:32 GMT"} })
16452025-08-26T21:46:33.329ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch = Switch0
16462025-08-26T21:46:33.329ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
16472025-08-26T21:46:33.329ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "0b98f5f1-cca0-413d-81e0-a64db1883439", "content-length": "682", "date": "Tue, 26 Aug 2025 21:46:33 GMT"} })
16482025-08-26T21:46:33.329ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
16492025-08-26T21:46:33.330ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
16502025-08-26T21:46:33.352ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
16512025-08-26T21:46:33.353ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
16522025-08-26T21:46:33.356ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
16532025-08-26T21:46:33.356ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
16542025-08-26T21:46:33.365ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
16552025-08-26T21:46:33.365ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
16562025-08-26T21:46:37.846ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 8 reason = Timeout
16572025-08-26T21:46:37.846ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
16582025-08-26T21:46:37.846ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
16592025-08-26T21:46:37.849ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16602025-08-26T21:46:37.852ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
16612025-08-26T21:46:37.852ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
16622025-08-26T21:46:37.852ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
16632025-08-26T21:46:37.854ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
16642025-08-26T21:46:37.854ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 8.646065ms iteration = 8 status = {"chicken_switches_updated":false}
16652025-08-26T21:46:37.866ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 5.00707969s iteration = 2 status = {}
16662025-08-26T21:46:37.873ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 5.013976385s iteration = 2 status = {"errors":[],"garbage_collect_requested":[]}
16672025-08-26T21:46:37.969ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
16682025-08-26T21:46:37.969ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
16692025-08-26T21:46:38.022ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
16702025-08-26T21:46:38.022ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
16712025-08-26T21:46:38.024ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16722025-08-26T21:46:38.026ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
16732025-08-26T21:46:38.026ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
16742025-08-26T21:46:38.026ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
16752025-08-26T21:46:38.028ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16762025-08-26T21:46:38.030ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
16772025-08-26T21:46:38.030ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
16782025-08-26T21:46:38.030ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
16792025-08-26T21:46:38.065ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
16802025-08-26T21:46:38.065ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "f4ecef3b-3373-41ca-96f0-a8fc07a02483", "content-length": "691", "date": "Tue, 26 Aug 2025 21:46:38 GMT"} })
16812025-08-26T21:46:38.065ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
16822025-08-26T21:46:38.065ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
16832025-08-26T21:46:38.065ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
16842025-08-26T21:46:38.067ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 5.20803899s iteration = 2 status = {}
16852025-08-26T21:46:42.846ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 9 reason = Timeout
16862025-08-26T21:46:42.846ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
16872025-08-26T21:46:42.847ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
16882025-08-26T21:46:42.849ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16892025-08-26T21:46:42.852ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
16902025-08-26T21:46:42.852ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
16912025-08-26T21:46:42.852ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
16922025-08-26T21:46:42.854ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
16932025-08-26T21:46:42.854ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.624131ms iteration = 9 status = {"chicken_switches_updated":false}
16942025-08-26T21:46:42.859ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 5 reason = Timeout
16952025-08-26T21:46:42.859ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
16962025-08-26T21:46:42.859ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
16972025-08-26T21:46:42.861ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
16982025-08-26T21:46:42.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
16992025-08-26T21:46:42.863ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.237636ms iteration = 5 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
17002025-08-26T21:46:42.870ZWARNnexus (BackgroundTasks): failed to read target blueprint background_task = blueprint_loader error = Internal Error: unexpected database error: no inbound stream connection file = nexus/src/app/background/tasks/blueprint_load.rs:69 original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC
17012025-08-26T21:46:42.870ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 10.010835124s iteration = 4 status = {"error":"failed to read target blueprint: Internal Error: unexpected database error: no inbound stream connection"}
17022025-08-26T21:46:42.870ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 5 reason = Timeout
17032025-08-26T21:46:42.870ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
17042025-08-26T21:46:42.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
17052025-08-26T21:46:42.872ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17062025-08-26T21:46:42.875ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
17072025-08-26T21:46:42.875ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
17082025-08-26T21:46:42.875ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
17092025-08-26T21:46:42.877ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
17102025-08-26T21:46:42.877ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
17112025-08-26T21:46:42.879ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17122025-08-26T21:46:42.882ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
17132025-08-26T21:46:42.882ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
17142025-08-26T21:46:42.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
17152025-08-26T21:46:42.996ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
17162025-08-26T21:46:42.996ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 125.718964ms iteration = 5 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
17172025-08-26T21:46:47.846ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 10 reason = Timeout
17182025-08-26T21:46:47.846ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
17192025-08-26T21:46:47.846ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
17202025-08-26T21:46:47.848ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17212025-08-26T21:46:47.851ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
17222025-08-26T21:46:47.851ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
17232025-08-26T21:46:47.851ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
17242025-08-26T21:46:47.852ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
17252025-08-26T21:46:47.852ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 6.441516ms iteration = 10 status = {"chicken_switches_updated":false}
17262025-08-26T21:46:52.848ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 11 reason = Timeout
17272025-08-26T21:46:52.848ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
17282025-08-26T21:46:52.848ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
17292025-08-26T21:46:52.850ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17302025-08-26T21:46:52.853ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
17312025-08-26T21:46:52.853ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
17322025-08-26T21:46:52.853ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
17332025-08-26T21:46:52.855ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
17342025-08-26T21:46:52.855ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.732215ms iteration = 11 status = {"chicken_switches_updated":false}
17352025-08-26T21:46:52.861ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 6 reason = Timeout
17362025-08-26T21:46:52.861ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
17372025-08-26T21:46:52.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
17382025-08-26T21:46:52.863ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
17392025-08-26T21:46:52.863ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
17402025-08-26T21:46:52.864ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.528359ms iteration = 6 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
17412025-08-26T21:46:52.873ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 6 reason = Timeout
17422025-08-26T21:46:52.873ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
17432025-08-26T21:46:52.874ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
17442025-08-26T21:46:52.875ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17452025-08-26T21:46:52.878ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
17462025-08-26T21:46:52.878ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
17472025-08-26T21:46:52.878ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
17482025-08-26T21:46:52.879ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
17492025-08-26T21:46:52.880ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
17502025-08-26T21:46:52.881ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17512025-08-26T21:46:52.884ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
17522025-08-26T21:46:52.884ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
17532025-08-26T21:46:52.884ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
17542025-08-26T21:46:52.897ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
17552025-08-26T21:46:52.897ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 24.318898ms iteration = 6 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
17562025-08-26T21:46:57.848ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 12 reason = Timeout
17572025-08-26T21:46:57.848ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
17582025-08-26T21:46:57.849ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
17592025-08-26T21:46:57.851ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17602025-08-26T21:46:57.854ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
17612025-08-26T21:46:57.854ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
17622025-08-26T21:46:57.854ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
17632025-08-26T21:46:57.856ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
17642025-08-26T21:46:57.856ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.643462ms iteration = 12 status = {"chicken_switches_updated":false}
17652025-08-26T21:47:02.835ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 3 reason = Timeout
17662025-08-26T21:47:02.835ZWARNnexus (BackgroundTasks): DNS propagation: skipped background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:69 reason = no config
17672025-08-26T21:47:02.835ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 20.82\u{b5}s iteration = 3 status = {"error":"no config"}
17682025-08-26T21:47:02.835ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 2 reason = Timeout
17692025-08-26T21:47:02.835ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 3 reason = Timeout
17702025-08-26T21:47:02.835ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
17712025-08-26T21:47:02.835ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
17722025-08-26T21:47:02.835ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
17732025-08-26T21:47:02.835ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 3 reason = Timeout
17742025-08-26T21:47:02.835ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 3 reason = Timeout
17752025-08-26T21:47:02.835ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
17762025-08-26T21:47:02.835ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 2 reason = Timeout
17772025-08-26T21:47:02.835ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 2 reason = Timeout
17782025-08-26T21:47:02.835ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
17792025-08-26T21:47:02.835ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 2 reason = Timeout
17802025-08-26T21:47:02.835ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 2 reason = Timeout
17812025-08-26T21:47:02.835ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
17822025-08-26T21:47:02.835ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
17832025-08-26T21:47:02.835ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 3 reason = Timeout
17842025-08-26T21:47:02.835ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 2 reason = Timeout
17852025-08-26T21:47:02.835ZWARNnexus (BackgroundTasks): DNS propagation: skipped background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:69 reason = no config
17862025-08-26T21:47:02.835ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-08-26 21:37:02.835260555 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
17872025-08-26T21:47:02.835ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 18.95\u{b5}s iteration = 3 status = {"error":"no config"}
17882025-08-26T21:47:02.835ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
17892025-08-26T21:47:02.835ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
17902025-08-26T21:47:02.835ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
17912025-08-26T21:47:02.835ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
17922025-08-26T21:47:02.835ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
17932025-08-26T21:47:02.835ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
17942025-08-26T21:47:02.835ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
17952025-08-26T21:47:02.835ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("852fd1a3-de9a-47e8-a667-5e8dab880ad3.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("c68f1501-294c-4874-92ca-8fc0546a5870.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("d741a72f-8745-4723-9e80-e1e72987266b.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("d741a72f-8745-4723-9e80-e1e72987266b.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:3::1)) }], valid_until: Instant { tv_sec: 926, tv_nsec: 692880874 } })
17962025-08-26T21:47:02.835ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("60abf731-31e1-4111-a3e2-65da89eaed87.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("84c849eb-0949-41d9-ad02-9a5509144564.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("84c849eb-0949-41d9-ad02-9a5509144564.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::8)) }], valid_until: Instant { tv_sec: 926, tv_nsec: 692885754 } })
17972025-08-26T21:47:02.836ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
17982025-08-26T21:47:02.836ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
17992025-08-26T21:47:02.836ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.252868ms iteration = 2 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
18002025-08-26T21:47:02.836ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } dns_group = internal
18012025-08-26T21:47:02.836ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.258788ms iteration = 2 status = {"addresses":["[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:3::1]:5353"]}
18022025-08-26T21:47:02.836ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
18032025-08-26T21:47:02.836ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
18042025-08-26T21:47:02.837ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
18052025-08-26T21:47:02.837ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 3 reason = Timeout
18062025-08-26T21:47:02.837ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:91 reason = blueprint disabled target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8
18072025-08-26T21:47:02.837ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 14.41\u{b5}s iteration = 3 status = {"enabled":false,"target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8"}
18082025-08-26T21:47:02.837ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18092025-08-26T21:47:02.838ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 926, tv_nsec: 695311999 } })
18102025-08-26T21:47:02.838ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 926, tv_nsec: 695374990 } })
18112025-08-26T21:47:02.838ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1205
18122025-08-26T21:47:02.838ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18132025-08-26T21:47:02.838ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-08-26 21:37:02.835260555 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
18142025-08-26T21:47:02.838ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 3.367406ms iteration = 2 status = {"expiration":"2025-08-26T21:37:02.835260555Z","pruned":[]}
18152025-08-26T21:47:02.838ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1205
18162025-08-26T21:47:02.839ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18172025-08-26T21:47:02.840ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 4.90618ms iteration = 3 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
18182025-08-26T21:47:02.840ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
18192025-08-26T21:47:02.840ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
18202025-08-26T21:47:02.840ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
18212025-08-26T21:47:02.843ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
18222025-08-26T21:47:02.843ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
18232025-08-26T21:47:02.843ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
18242025-08-26T21:47:02.843ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
18252025-08-26T21:47:02.845ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18262025-08-26T21:47:02.848ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
18272025-08-26T21:47:02.848ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
18282025-08-26T21:47:02.848ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
18292025-08-26T21:47:02.849ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 13 reason = Timeout
18302025-08-26T21:47:02.849ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
18312025-08-26T21:47:02.849ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
18322025-08-26T21:47:02.850ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18332025-08-26T21:47:02.853ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
18342025-08-26T21:47:02.853ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
18352025-08-26T21:47:02.853ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
18362025-08-26T21:47:02.855ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
18372025-08-26T21:47:02.855ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 5.75354ms iteration = 13 status = {"chicken_switches_updated":false}
18382025-08-26T21:47:02.862ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 4 reason = Timeout
18392025-08-26T21:47:02.862ZDEBGnexus (BackgroundTasks): blueprint planning disabled, doing nothing background_task = blueprint_planner
18402025-08-26T21:47:02.862ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 11.82\u{b5}s iteration = 4 status = "Disabled"
18412025-08-26T21:47:02.862ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 3 reason = Timeout
18422025-08-26T21:47:02.862ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 3 reason = Timeout
18432025-08-26T21:47:02.862ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
18442025-08-26T21:47:02.862ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
18452025-08-26T21:47:02.862ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 2 reason = Timeout
18462025-08-26T21:47:02.862ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
18472025-08-26T21:47:02.862ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
18482025-08-26T21:47:02.862ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 3 reason = Timeout
18492025-08-26T21:47:02.862ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 7 reason = Timeout
18502025-08-26T21:47:02.862ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
18512025-08-26T21:47:02.862ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 4 reason = Timeout
18522025-08-26T21:47:02.862ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
18532025-08-26T21:47:02.862ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
18542025-08-26T21:47:02.862ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 2 reason = Timeout
18552025-08-26T21:47:02.862ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:53
18562025-08-26T21:47:02.862ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 3 reason = Timeout
18572025-08-26T21:47:02.862ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
18582025-08-26T21:47:02.862ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
18592025-08-26T21:47:02.862ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 3 reason = Timeout
18602025-08-26T21:47:02.862ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 2 reason = Timeout
18612025-08-26T21:47:02.862ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
18622025-08-26T21:47:02.862ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
18632025-08-26T21:47:02.862ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 3 reason = Timeout
18642025-08-26T21:47:02.862ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
18652025-08-26T21:47:02.862ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 3 reason = Timeout
18662025-08-26T21:47:02.862ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
18672025-08-26T21:47:02.862ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
18682025-08-26T21:47:02.862ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 3 reason = Timeout
18692025-08-26T21:47:02.862ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
18702025-08-26T21:47:02.862ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 2 reason = Timeout
18712025-08-26T21:47:02.862ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
18722025-08-26T21:47:02.862ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 3 reason = Timeout
18732025-08-26T21:47:02.862ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
18742025-08-26T21:47:02.862ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 3 reason = Timeout
18752025-08-26T21:47:02.862ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
18762025-08-26T21:47:02.862ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 3 reason = Timeout
18772025-08-26T21:47:02.862ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
18782025-08-26T21:47:02.862ZDEBGnexus (BackgroundTasks): activating background_task = alert_dispatcher iteration = 2 reason = Timeout
18792025-08-26T21:47:02.862ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
18802025-08-26T21:47:02.862ZDEBGnexus (BackgroundTasks): activating background_task = webhook_deliverator iteration = 2 reason = Timeout
18812025-08-26T21:47:02.862ZDEBGnexus (BackgroundTasks): roles background_task = webhook_deliverator roles = RoleSet { roles: {} }
18822025-08-26T21:47:02.862ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 3 reason = Timeout
18832025-08-26T21:47:02.862ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
18842025-08-26T21:47:02.862ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 3 reason = Timeout
18852025-08-26T21:47:02.862ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
18862025-08-26T21:47:02.862ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 3 reason = Timeout
18872025-08-26T21:47:02.862ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 9.56\u{b5}s iteration = 3 status = {"disabled":true,"errors":[],"sps":[]}
18882025-08-26T21:47:02.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
18892025-08-26T21:47:02.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
18902025-08-26T21:47:02.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
18912025-08-26T21:47:02.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
18922025-08-26T21:47:02.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
18932025-08-26T21:47:02.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
18942025-08-26T21:47:02.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
18952025-08-26T21:47:02.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
18962025-08-26T21:47:02.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
18972025-08-26T21:47:02.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
18982025-08-26T21:47:02.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
18992025-08-26T21:47:02.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
19002025-08-26T21:47:02.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
19012025-08-26T21:47:02.863ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
19022025-08-26T21:47:02.863ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
19032025-08-26T21:47:02.863ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
19042025-08-26T21:47:02.863ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = alert_dispatcher resource = Database result = Ok(())
19052025-08-26T21:47:02.863ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = webhook_deliverator resource = Database result = Ok(())
19062025-08-26T21:47:02.863ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
19072025-08-26T21:47:02.863ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
19082025-08-26T21:47:02.863ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
19092025-08-26T21:47:02.864ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19102025-08-26T21:47:02.864ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19112025-08-26T21:47:02.864ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19122025-08-26T21:47:02.864ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
19132025-08-26T21:47:02.865ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 2.709971ms iteration = 3 status = {"errors":[],"finish_invoked_ok":[]}
19142025-08-26T21:47:02.865ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
19152025-08-26T21:47:02.865ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19162025-08-26T21:47:02.865ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 3.158361ms iteration = 2 status = {"errors":[],"found_port_ok":[]}
19172025-08-26T21:47:02.866ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19182025-08-26T21:47:02.868ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
19192025-08-26T21:47:02.868ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
19202025-08-26T21:47:02.868ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
19212025-08-26T21:47:02.868ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
19222025-08-26T21:47:02.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
19232025-08-26T21:47:02.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
19242025-08-26T21:47:02.869ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
19252025-08-26T21:47:02.869ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
19262025-08-26T21:47:02.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
19272025-08-26T21:47:02.870ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
19282025-08-26T21:47:02.870ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
19292025-08-26T21:47:02.870ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
19302025-08-26T21:47:02.870ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
19312025-08-26T21:47:02.870ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
19322025-08-26T21:47:02.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
19332025-08-26T21:47:02.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = alert_dispatcher resource = Database result = Ok(())
19342025-08-26T21:47:02.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
19352025-08-26T21:47:02.871ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
19362025-08-26T21:47:02.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
19372025-08-26T21:47:02.871ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19382025-08-26T21:47:02.871ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
19392025-08-26T21:47:02.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
19402025-08-26T21:47:02.873ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 7 reason = Timeout
19412025-08-26T21:47:02.873ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19422025-08-26T21:47:02.873ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
19432025-08-26T21:47:02.873ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
19442025-08-26T21:47:02.875ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
19452025-08-26T21:47:02.875ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
19462025-08-26T21:47:02.875ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
19472025-08-26T21:47:02.875ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 13.468734ms iteration = 3 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
19482025-08-26T21:47:02.876ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
19492025-08-26T21:47:02.876ZINFOnexus (BackgroundTasks): all webhook delivery tasks completed background_task = webhook_deliverator file = nexus/src/app/background/tasks/webhook_deliverator.rs:204 num_receivers = 0
19502025-08-26T21:47:02.876ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
19512025-08-26T21:47:02.876ZDEBGnexus (BackgroundTasks): activation complete background_task = webhook_deliverator elapsed = 13.668418ms iteration = 2 status = {"by_rx":{},"error":null}
19522025-08-26T21:47:02.876ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
19532025-08-26T21:47:02.877ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19542025-08-26T21:47:02.882ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
19552025-08-26T21:47:02.882ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
19562025-08-26T21:47:02.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
19572025-08-26T21:47:02.892ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
19582025-08-26T21:47:02.892ZDEBGnexus (BackgroundTasks): activation complete background_task = alert_dispatcher elapsed = 29.947275ms iteration = 2 status = {"dispatched":[],"errors":[],"glob_version":"183.0.0","globs_reprocessed":{},"no_receivers":[]}
19592025-08-26T21:47:02.892ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
19602025-08-26T21:47:02.899ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
19612025-08-26T21:47:02.899ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
19622025-08-26T21:47:02.899ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
19632025-08-26T21:47:02.899ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
19642025-08-26T21:47:02.899ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
19652025-08-26T21:47:02.900ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
19662025-08-26T21:47:02.901ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
19672025-08-26T21:47:02.909ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
19682025-08-26T21:47:02.909ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
19692025-08-26T21:47:02.909ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
19702025-08-26T21:47:02.909ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
19712025-08-26T21:47:02.909ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1213 zone_address = fd00:1122:3344:101::2
19722025-08-26T21:47:02.909ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
19732025-08-26T21:47:02.909ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
19742025-08-26T21:47:02.910ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "2feb6c07-0a8e-4962-be86-89860ccdf9e3", "content-length": "26", "date": "Tue, 26 Aug 2025 21:47:02 GMT"} })
19752025-08-26T21:47:02.910ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1216 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
19762025-08-26T21:47:02.911ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
19772025-08-26T21:47:02.912ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19782025-08-26T21:47:02.913ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
19792025-08-26T21:47:02.914ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19802025-08-26T21:47:02.914ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
19812025-08-26T21:47:02.915ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19822025-08-26T21:47:02.916ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19832025-08-26T21:47:02.918ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
19842025-08-26T21:47:02.918ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
19852025-08-26T21:47:02.918ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
19862025-08-26T21:47:02.918ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
19872025-08-26T21:47:02.918ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
19882025-08-26T21:47:02.918ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
19892025-08-26T21:47:02.919ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
19902025-08-26T21:47:02.919ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
19912025-08-26T21:47:02.920ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
19922025-08-26T21:47:02.920ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
19932025-08-26T21:47:02.920ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
19942025-08-26T21:47:02.921ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
19952025-08-26T21:47:02.921ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
19962025-08-26T21:47:02.922ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19972025-08-26T21:47:02.922ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
19982025-08-26T21:47:02.922ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
19992025-08-26T21:47:02.923ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
20002025-08-26T21:47:02.923ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
20012025-08-26T21:47:02.923ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
20022025-08-26T21:47:02.923ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 60.560015ms iteration = 3 status = {"errors":[],"garbage_collect_requested":[]}
20032025-08-26T21:47:02.923ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
20042025-08-26T21:47:02.924ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20052025-08-26T21:47:02.924ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = dfb404ab-acb8-43de-afe8-b429af0c161d dns_zone_name = oxide.test found_so_far = 0 version = 2
20062025-08-26T21:47:02.924ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
20072025-08-26T21:47:02.924ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
20082025-08-26T21:47:02.925ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
20092025-08-26T21:47:02.925ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
20102025-08-26T21:47:02.925ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
20112025-08-26T21:47:02.925ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
20122025-08-26T21:47:02.925ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
20132025-08-26T21:47:02.925ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
20142025-08-26T21:47:02.925ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
20152025-08-26T21:47:02.926ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20162025-08-26T21:47:02.926ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1213 zone_address = fd00:1122:3344:101::2
20172025-08-26T21:47:02.926ZDEBGnexus (MgsClient): client request background_task = nat_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
20182025-08-26T21:47:02.926ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
20192025-08-26T21:47:02.926ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
20202025-08-26T21:47:02.926ZDEBGnexus (MgsClient): client response background_task = nat_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "55fb5114-e4b3-4191-a896-e24353ac9af7", "content-length": "26", "date": "Tue, 26 Aug 2025 21:47:02 GMT"} })
20212025-08-26T21:47:02.926ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1216 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
20222025-08-26T21:47:02.926ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
20232025-08-26T21:47:02.928ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
20242025-08-26T21:47:02.928ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 926, tv_nsec: 785856720 } })
20252025-08-26T21:47:02.929ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1205 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
20262025-08-26T21:47:02.930ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
20272025-08-26T21:47:02.930ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
20282025-08-26T21:47:02.930ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
20292025-08-26T21:47:02.932ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 69.559508ms iteration = 3 status = {"errors":[],"requests_created_ok":[]}
20302025-08-26T21:47:02.932ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 69.799643ms iteration = 7 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
20312025-08-26T21:47:02.932ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:550 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
20322025-08-26T21:47:02.932ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 70.368176ms iteration = 3 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
20332025-08-26T21:47:02.933ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = dfb404ab-acb8-43de-afe8-b429af0c161d dns_zone_name = oxide.test found_so_far = 4 version = 2
20342025-08-26T21:47:02.933ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
20352025-08-26T21:47:02.933ZINFOnexus (BackgroundTasks): found latest generation (first find) background_task = dns_config_external dns_group = external file = nexus/src/app/background/tasks/dns_config.rs:87 generation = 2
20362025-08-26T21:47:02.933ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 97.825805ms iteration = 2 status = {"generation":2}
20372025-08-26T21:47:02.933ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 4 reason = Dependency
20382025-08-26T21:47:02.934ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
20392025-08-26T21:47:02.934ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:170 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
20402025-08-26T21:47:02.934ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 71.936881ms iteration = 2 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
20412025-08-26T21:47:02.935ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
20422025-08-26T21:47:02.935ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
20432025-08-26T21:47:02.945ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
20442025-08-26T21:47:02.945ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 71.648095ms iteration = 7 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
20452025-08-26T21:47:02.946ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
20462025-08-26T21:47:02.947ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
20472025-08-26T21:47:02.957ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20482025-08-26T21:47:02.958ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
20492025-08-26T21:47:02.958ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:138 found = 0 not_ready_to_be_deleted = 0
20502025-08-26T21:47:02.958ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 96.27352ms iteration = 2 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
20512025-08-26T21:47:02.959ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
20522025-08-26T21:47:02.959ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
20532025-08-26T21:47:02.960ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
20542025-08-26T21:47:02.967ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
20552025-08-26T21:47:02.967ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
20562025-08-26T21:47:02.974ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
20572025-08-26T21:47:02.974ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
20582025-08-26T21:47:02.974ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 111.745799ms iteration = 3 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
20592025-08-26T21:47:02.977ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
20602025-08-26T21:47:02.977ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 8e720f02-dde8-4423-a6e3-91698373805a dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
20612025-08-26T21:47:02.977ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
20622025-08-26T21:47:02.977ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
20632025-08-26T21:47:02.979ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20642025-08-26T21:47:02.982ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
20652025-08-26T21:47:02.982ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
20662025-08-26T21:47:02.982ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
20672025-08-26T21:47:02.987ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 125.191932ms iteration = 3 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
20682025-08-26T21:47:02.989ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 8e720f02-dde8-4423-a6e3-91698373805a dns_zone_name = control-plane.oxide.internal found_so_far = 59 version = 1
20692025-08-26T21:47:02.989ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
20702025-08-26T21:47:02.989ZINFOnexus (BackgroundTasks): found latest generation (first find) background_task = dns_config_internal dns_group = internal file = nexus/src/app/background/tasks/dns_config.rs:87 generation = 1
20712025-08-26T21:47:02.989ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 154.275748ms iteration = 2 status = {"generation":1}
20722025-08-26T21:47:02.989ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 4 reason = Dependency
20732025-08-26T21:47:03.005ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:407
20742025-08-26T21:47:03.005ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 143.503945ms iteration = 3 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
20752025-08-26T21:47:03.012ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 150.037223ms iteration = 3 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
20762025-08-26T21:47:03.016ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
20772025-08-26T21:47:03.017ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
20782025-08-26T21:47:03.017ZINFOnexus (BackgroundTasks): found Silo/DNS/TLS config (initial) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-08-26T21:45:59.866217Z, time_modified: 2025-08-26T21:45:59.866217Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 9396aa55-de6f-4d6f-96db-d28e1c76fa9c, db_silo: Silo { identity: SiloIdentity { id: 9396aa55-de6f-4d6f-96db-d28e1c76fa9c, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-08-26T21:46:05.245521Z, time_modified: 2025-08-26T21:46:05.245521Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "e8529d720e4b0de099753dcdb373abb870af04baacd7c7dd850f5254858081af" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 9396aa55-de6f-4d6f-96db-d28e1c76fa9c, db_silo: Silo { identity: SiloIdentity { id: 9396aa55-de6f-4d6f-96db-d28e1c76fa9c, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-08-26T21:46:05.245521Z, time_modified: 2025-08-26T21:46:05.245521Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "e8529d720e4b0de099753dcdb373abb870af04baacd7c7dd850f5254858081af" }] }) } file = nexus/src/app/background/tasks/external_endpoints.rs:77
20792025-08-26T21:47:03.017ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 182.410822ms iteration = 2 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"9396aa55-de6f-4d6f-96db-d28e1c76fa9c","tls_certs":["e8529d720e4b0de099753dcdb373abb870af04baacd7c7dd850f5254858081af"]}},"default_endpoint":{"silo_id":"9396aa55-de6f-4d6f-96db-d28e1c76fa9c","tls_certs":["e8529d720e4b0de099753dcdb373abb870af04baacd7c7dd850f5254858081af"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
20802025-08-26T21:47:03.030ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
20812025-08-26T21:47:03.030ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
20822025-08-26T21:47:03.048ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 186.400902ms iteration = 4 status = {"physical_disks_added":0}
20832025-08-26T21:47:03.068ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = / versions = all
20842025-08-26T21:47:03.068ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /access versions = all
20852025-08-26T21:47:03.068ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /assets/{path} versions = all
20862025-08-26T21:47:03.068ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /device/auth versions = all
20872025-08-26T21:47:03.068ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /device/confirm versions = all
20882025-08-26T21:47:03.068ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /device/success versions = all
20892025-08-26T21:47:03.068ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /device/token versions = all
20902025-08-26T21:47:03.068ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /device/verify versions = all
20912025-08-26T21:47:03.068ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /experimental/v1/probes versions = all
20922025-08-26T21:47:03.068ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /experimental/v1/probes versions = all
20932025-08-26T21:47:03.068ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /experimental/v1/probes/{probe} versions = all
20942025-08-26T21:47:03.068ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /experimental/v1/probes/{probe} versions = all
20952025-08-26T21:47:03.068ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /experimental/v1/system/support-bundles versions = all
20962025-08-26T21:47:03.068ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /experimental/v1/system/support-bundles versions = all
20972025-08-26T21:47:03.068ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /experimental/v1/system/support-bundles/{bundle_id} versions = all
20982025-08-26T21:47:03.068ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /experimental/v1/system/support-bundles/{bundle_id} versions = all
20992025-08-26T21:47:03.068ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = PUT path = /experimental/v1/system/support-bundles/{bundle_id} versions = all
21002025-08-26T21:47:03.068ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /experimental/v1/system/support-bundles/{bundle_id}/download versions = all
21012025-08-26T21:47:03.068ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = HEAD path = /experimental/v1/system/support-bundles/{bundle_id}/download versions = all
21022025-08-26T21:47:03.068ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /experimental/v1/system/support-bundles/{bundle_id}/download/{file} versions = all
21032025-08-26T21:47:03.068ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = HEAD path = /experimental/v1/system/support-bundles/{bundle_id}/download/{file} versions = all
21042025-08-26T21:47:03.068ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /experimental/v1/system/support-bundles/{bundle_id}/index versions = all
21052025-08-26T21:47:03.068ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /images versions = all
21062025-08-26T21:47:03.068ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /login versions = all
21072025-08-26T21:47:03.068ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /login/{silo_name}/local versions = all
21082025-08-26T21:47:03.068ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /login/{silo_name}/saml/{provider_name} versions = all
21092025-08-26T21:47:03.068ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /login/{silo_name}/saml/{provider_name} versions = all
21102025-08-26T21:47:03.068ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /login/{silo_name}/saml/{provider_name}/redirect versions = all
21112025-08-26T21:47:03.068ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /lookup/{path} versions = all
21122025-08-26T21:47:03.068ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /projects/{path} versions = all
21132025-08-26T21:47:03.068ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /projects-new versions = all
21142025-08-26T21:47:03.068ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /settings/{path} versions = all
21152025-08-26T21:47:03.068ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /system/{path} versions = all
21162025-08-26T21:47:03.068ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /utilization versions = all
21172025-08-26T21:47:03.068ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/affinity-groups versions = all
21182025-08-26T21:47:03.068ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/affinity-groups versions = all
21192025-08-26T21:47:03.068ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/affinity-groups/{affinity_group} versions = all
21202025-08-26T21:47:03.068ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/affinity-groups/{affinity_group} versions = all
21212025-08-26T21:47:03.068ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = PUT path = /v1/affinity-groups/{affinity_group} versions = all
21222025-08-26T21:47:03.068ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/affinity-groups/{affinity_group}/members versions = all
21232025-08-26T21:47:03.068ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/affinity-groups/{affinity_group}/members/instance/{instance} versions = all
21242025-08-26T21:47:03.068ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/affinity-groups/{affinity_group}/members/instance/{instance} versions = all
21252025-08-26T21:47:03.068ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/affinity-groups/{affinity_group}/members/instance/{instance} versions = all
21262025-08-26T21:47:03.068ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/alert-classes versions = all
21272025-08-26T21:47:03.068ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/alert-receivers versions = all
21282025-08-26T21:47:03.068ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/alert-receivers/{receiver} versions = all
21292025-08-26T21:47:03.068ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/alert-receivers/{receiver} versions = all
21302025-08-26T21:47:03.068ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/alert-receivers/{receiver}/deliveries versions = all
21312025-08-26T21:47:03.068ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/alert-receivers/{receiver}/probe versions = all
21322025-08-26T21:47:03.068ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/alert-receivers/{receiver}/subscriptions versions = all
21332025-08-26T21:47:03.068ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/alert-receivers/{receiver}/subscriptions/{subscription} versions = all
21342025-08-26T21:47:03.068ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/alerts/{alert_id}/resend versions = all
21352025-08-26T21:47:03.068ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/anti-affinity-groups versions = all
21362025-08-26T21:47:03.068ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/anti-affinity-groups versions = all
21372025-08-26T21:47:03.068ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/anti-affinity-groups/{anti_affinity_group} versions = all
21382025-08-26T21:47:03.068ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/anti-affinity-groups/{anti_affinity_group} versions = all
21392025-08-26T21:47:03.068ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = PUT path = /v1/anti-affinity-groups/{anti_affinity_group} versions = all
21402025-08-26T21:47:03.068ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/anti-affinity-groups/{anti_affinity_group}/members versions = all
21412025-08-26T21:47:03.068ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/anti-affinity-groups/{anti_affinity_group}/members/instance/{instance} versions = all
21422025-08-26T21:47:03.068ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/anti-affinity-groups/{anti_affinity_group}/members/instance/{instance} versions = all
21432025-08-26T21:47:03.068ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/anti-affinity-groups/{anti_affinity_group}/members/instance/{instance} versions = all
21442025-08-26T21:47:03.068ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/auth-settings versions = all
21452025-08-26T21:47:03.068ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = PUT path = /v1/auth-settings versions = all
21462025-08-26T21:47:03.068ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/certificates versions = all
21472025-08-26T21:47:03.068ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/certificates versions = all
21482025-08-26T21:47:03.068ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/certificates/{certificate} versions = all
21492025-08-26T21:47:03.068ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/certificates/{certificate} versions = all
21502025-08-26T21:47:03.068ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/disks versions = all
21512025-08-26T21:47:03.068ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/disks versions = all
21522025-08-26T21:47:03.068ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/disks/{disk} versions = all
21532025-08-26T21:47:03.068ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/disks/{disk} versions = all
21542025-08-26T21:47:03.068ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/disks/{disk}/bulk-write versions = all
21552025-08-26T21:47:03.068ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/disks/{disk}/bulk-write-start versions = all
21562025-08-26T21:47:03.068ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/disks/{disk}/bulk-write-stop versions = all
21572025-08-26T21:47:03.068ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/disks/{disk}/finalize versions = all
21582025-08-26T21:47:03.068ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/floating-ips versions = all
21592025-08-26T21:47:03.068ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/floating-ips versions = all
21602025-08-26T21:47:03.068ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/floating-ips/{floating_ip} versions = all
21612025-08-26T21:47:03.068ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/floating-ips/{floating_ip} versions = all
21622025-08-26T21:47:03.068ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = PUT path = /v1/floating-ips/{floating_ip} versions = all
21632025-08-26T21:47:03.068ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/floating-ips/{floating_ip}/attach versions = all
21642025-08-26T21:47:03.068ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/floating-ips/{floating_ip}/detach versions = all
21652025-08-26T21:47:03.068ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/groups versions = all
21662025-08-26T21:47:03.068ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/groups/{group_id} versions = all
21672025-08-26T21:47:03.068ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/images versions = all
21682025-08-26T21:47:03.068ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/images versions = all
21692025-08-26T21:47:03.068ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/images/{image} versions = all
21702025-08-26T21:47:03.068ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/images/{image} versions = all
21712025-08-26T21:47:03.068ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/images/{image}/demote versions = all
21722025-08-26T21:47:03.068ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/images/{image}/promote versions = all
21732025-08-26T21:47:03.068ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/instances versions = all
21742025-08-26T21:47:03.068ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/instances versions = all
21752025-08-26T21:47:03.068ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/instances/{instance} versions = all
21762025-08-26T21:47:03.068ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/instances/{instance} versions = all
21772025-08-26T21:47:03.068ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = PUT path = /v1/instances/{instance} versions = all
21782025-08-26T21:47:03.068ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/instances/{instance}/affinity-groups versions = all
21792025-08-26T21:47:03.068ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/instances/{instance}/anti-affinity-groups versions = all
21802025-08-26T21:47:03.068ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/instances/{instance}/disks versions = all
21812025-08-26T21:47:03.068ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/instances/{instance}/disks/attach versions = all
21822025-08-26T21:47:03.068ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/instances/{instance}/disks/detach versions = all
21832025-08-26T21:47:03.068ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/instances/{instance}/external-ips versions = all
21842025-08-26T21:47:03.068ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/instances/{instance}/external-ips/ephemeral versions = all
21852025-08-26T21:47:03.068ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/instances/{instance}/external-ips/ephemeral versions = all
21862025-08-26T21:47:03.068ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/instances/{instance}/reboot versions = all
21872025-08-26T21:47:03.068ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/instances/{instance}/serial-console versions = all
21882025-08-26T21:47:03.068ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/instances/{instance}/serial-console/stream versions = all
21892025-08-26T21:47:03.068ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/instances/{instance}/ssh-public-keys versions = all
21902025-08-26T21:47:03.068ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/instances/{instance}/start versions = all
21912025-08-26T21:47:03.068ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/instances/{instance}/stop versions = all
21922025-08-26T21:47:03.068ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/internet-gateway-ip-addresses versions = all
21932025-08-26T21:47:03.068ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/internet-gateway-ip-addresses versions = all
21942025-08-26T21:47:03.068ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/internet-gateway-ip-addresses/{address} versions = all
21952025-08-26T21:47:03.068ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/internet-gateway-ip-pools versions = all
21962025-08-26T21:47:03.068ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/internet-gateway-ip-pools versions = all
21972025-08-26T21:47:03.068ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/internet-gateway-ip-pools/{pool} versions = all
21982025-08-26T21:47:03.068ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/internet-gateways versions = all
21992025-08-26T21:47:03.068ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/internet-gateways versions = all
22002025-08-26T21:47:03.068ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/internet-gateways/{gateway} versions = all
22012025-08-26T21:47:03.069ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/internet-gateways/{gateway} versions = all
22022025-08-26T21:47:03.069ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/ip-pools versions = all
22032025-08-26T21:47:03.069ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/ip-pools/{pool} versions = all
22042025-08-26T21:47:03.069ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/login/{silo_name}/local versions = all
22052025-08-26T21:47:03.069ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/logout versions = all
22062025-08-26T21:47:03.069ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/me versions = all
22072025-08-26T21:47:03.069ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/me/access-tokens versions = all
22082025-08-26T21:47:03.069ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/me/access-tokens/{token_id} versions = all
22092025-08-26T21:47:03.069ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/me/groups versions = all
22102025-08-26T21:47:03.069ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/me/ssh-keys versions = all
22112025-08-26T21:47:03.069ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/me/ssh-keys versions = all
22122025-08-26T21:47:03.069ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/me/ssh-keys/{ssh_key} versions = all
22132025-08-26T21:47:03.069ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/me/ssh-keys/{ssh_key} versions = all
22142025-08-26T21:47:03.069ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/metrics/{metric_name} versions = all
22152025-08-26T21:47:03.069ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/network-interfaces versions = all
22162025-08-26T21:47:03.069ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/network-interfaces versions = all
22172025-08-26T21:47:03.069ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/network-interfaces/{interface} versions = all
22182025-08-26T21:47:03.069ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/network-interfaces/{interface} versions = all
22192025-08-26T21:47:03.069ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = PUT path = /v1/network-interfaces/{interface} versions = all
22202025-08-26T21:47:03.069ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/ping versions = all
22212025-08-26T21:47:03.069ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/policy versions = all
22222025-08-26T21:47:03.069ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = PUT path = /v1/policy versions = all
22232025-08-26T21:47:03.069ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/projects versions = all
22242025-08-26T21:47:03.069ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/projects versions = all
22252025-08-26T21:47:03.069ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/projects/{project} versions = all
22262025-08-26T21:47:03.069ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/projects/{project} versions = all
22272025-08-26T21:47:03.069ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = PUT path = /v1/projects/{project} versions = all
22282025-08-26T21:47:03.069ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/projects/{project}/policy versions = all
22292025-08-26T21:47:03.069ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = PUT path = /v1/projects/{project}/policy versions = all
22302025-08-26T21:47:03.069ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/snapshots versions = all
22312025-08-26T21:47:03.069ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/snapshots versions = all
22322025-08-26T21:47:03.069ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/snapshots/{snapshot} versions = all
22332025-08-26T21:47:03.069ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/snapshots/{snapshot} versions = all
22342025-08-26T21:47:03.069ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/audit-log versions = all
22352025-08-26T21:47:03.069ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/hardware/disks versions = all
22362025-08-26T21:47:03.069ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/hardware/disks/{disk_id} versions = all
22372025-08-26T21:47:03.069ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/hardware/rack-switch-port/{rack_id}/{switch_location}/{port}/lldp/neighbors versions = all
22382025-08-26T21:47:03.069ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/hardware/racks versions = all
22392025-08-26T21:47:03.069ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/hardware/racks/{rack_id} versions = all
22402025-08-26T21:47:03.069ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/hardware/sleds versions = all
22412025-08-26T21:47:03.069ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/system/hardware/sleds versions = all
22422025-08-26T21:47:03.069ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/hardware/sleds/{sled_id} versions = all
22432025-08-26T21:47:03.069ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/hardware/sleds/{sled_id}/disks versions = all
22442025-08-26T21:47:03.069ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/hardware/sleds/{sled_id}/instances versions = all
22452025-08-26T21:47:03.069ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = PUT path = /v1/system/hardware/sleds/{sled_id}/provision-policy versions = all
22462025-08-26T21:47:03.069ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/hardware/sleds-uninitialized versions = all
22472025-08-26T21:47:03.069ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/hardware/switch-port versions = all
22482025-08-26T21:47:03.069ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/hardware/switch-port/{port}/lldp/config versions = all
22492025-08-26T21:47:03.069ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/system/hardware/switch-port/{port}/lldp/config versions = all
22502025-08-26T21:47:03.069ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/system/hardware/switch-port/{port}/settings versions = all
22512025-08-26T21:47:03.069ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/system/hardware/switch-port/{port}/settings versions = all
22522025-08-26T21:47:03.069ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/hardware/switch-port/{port}/status versions = all
22532025-08-26T21:47:03.069ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/hardware/switches versions = all
22542025-08-26T21:47:03.069ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/hardware/switches/{switch_id} versions = all
22552025-08-26T21:47:03.069ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/identity-providers versions = all
22562025-08-26T21:47:03.069ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/system/identity-providers/local/users versions = all
22572025-08-26T21:47:03.069ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/system/identity-providers/local/users/{user_id} versions = all
22582025-08-26T21:47:03.069ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/system/identity-providers/local/users/{user_id}/set-password versions = all
22592025-08-26T21:47:03.069ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/system/identity-providers/saml versions = all
22602025-08-26T21:47:03.069ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/identity-providers/saml/{provider} versions = all
22612025-08-26T21:47:03.069ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/ip-pools versions = all
22622025-08-26T21:47:03.069ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/system/ip-pools versions = all
22632025-08-26T21:47:03.069ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/system/ip-pools/{pool} versions = all
22642025-08-26T21:47:03.069ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/ip-pools/{pool} versions = all
22652025-08-26T21:47:03.069ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = PUT path = /v1/system/ip-pools/{pool} versions = all
22662025-08-26T21:47:03.069ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/ip-pools/{pool}/ranges versions = all
22672025-08-26T21:47:03.069ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/system/ip-pools/{pool}/ranges/add versions = all
22682025-08-26T21:47:03.069ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/system/ip-pools/{pool}/ranges/remove versions = all
22692025-08-26T21:47:03.069ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/ip-pools/{pool}/silos versions = all
22702025-08-26T21:47:03.069ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/system/ip-pools/{pool}/silos versions = all
22712025-08-26T21:47:03.069ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/system/ip-pools/{pool}/silos/{silo} versions = all
22722025-08-26T21:47:03.069ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = PUT path = /v1/system/ip-pools/{pool}/silos/{silo} versions = all
22732025-08-26T21:47:03.069ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/ip-pools/{pool}/utilization versions = all
22742025-08-26T21:47:03.069ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/ip-pools-service versions = all
22752025-08-26T21:47:03.069ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/ip-pools-service/ranges versions = all
22762025-08-26T21:47:03.069ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/system/ip-pools-service/ranges/add versions = all
22772025-08-26T21:47:03.069ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/system/ip-pools-service/ranges/remove versions = all
22782025-08-26T21:47:03.069ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/metrics/{metric_name} versions = all
22792025-08-26T21:47:03.069ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/networking/address-lot versions = all
22802025-08-26T21:47:03.069ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/system/networking/address-lot versions = all
22812025-08-26T21:47:03.069ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/system/networking/address-lot/{address_lot} versions = all
22822025-08-26T21:47:03.069ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/networking/address-lot/{address_lot} versions = all
22832025-08-26T21:47:03.069ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/networking/address-lot/{address_lot}/blocks versions = all
22842025-08-26T21:47:03.069ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/networking/allow-list versions = all
22852025-08-26T21:47:03.069ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = PUT path = /v1/system/networking/allow-list versions = all
22862025-08-26T21:47:03.069ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/system/networking/bfd-disable versions = all
22872025-08-26T21:47:03.069ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/system/networking/bfd-enable versions = all
22882025-08-26T21:47:03.069ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/networking/bfd-status versions = all
22892025-08-26T21:47:03.069ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/system/networking/bgp versions = all
22902025-08-26T21:47:03.069ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/networking/bgp versions = all
22912025-08-26T21:47:03.069ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/system/networking/bgp versions = all
22922025-08-26T21:47:03.069ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/networking/bgp-announce-set versions = all
22932025-08-26T21:47:03.069ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = PUT path = /v1/system/networking/bgp-announce-set versions = all
22942025-08-26T21:47:03.069ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/system/networking/bgp-announce-set/{announce_set} versions = all
22952025-08-26T21:47:03.069ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/networking/bgp-announce-set/{announce_set}/announcement versions = all
22962025-08-26T21:47:03.069ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/networking/bgp-exported versions = all
22972025-08-26T21:47:03.069ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/networking/bgp-message-history versions = all
22982025-08-26T21:47:03.069ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/networking/bgp-routes-ipv4 versions = all
22992025-08-26T21:47:03.069ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/networking/bgp-status versions = all
23002025-08-26T21:47:03.069ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/networking/inbound-icmp versions = all
23012025-08-26T21:47:03.069ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = PUT path = /v1/system/networking/inbound-icmp versions = all
23022025-08-26T21:47:03.069ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/networking/loopback-address versions = all
23032025-08-26T21:47:03.069ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/system/networking/loopback-address versions = all
23042025-08-26T21:47:03.069ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/system/networking/loopback-address/{rack_id}/{switch_location}/{address}/{subnet_mask} versions = all
23052025-08-26T21:47:03.069ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/system/networking/switch-port-settings versions = all
23062025-08-26T21:47:03.069ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/networking/switch-port-settings versions = all
23072025-08-26T21:47:03.069ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/system/networking/switch-port-settings versions = all
23082025-08-26T21:47:03.069ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/networking/switch-port-settings/{port} versions = all
23092025-08-26T21:47:03.069ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/policy versions = all
23102025-08-26T21:47:03.069ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = PUT path = /v1/system/policy versions = all
23112025-08-26T21:47:03.069ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/silo-quotas versions = all
23122025-08-26T21:47:03.069ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/silos versions = all
23132025-08-26T21:47:03.069ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/system/silos versions = all
23142025-08-26T21:47:03.069ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/system/silos/{silo} versions = all
23152025-08-26T21:47:03.069ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/silos/{silo} versions = all
23162025-08-26T21:47:03.069ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/silos/{silo}/ip-pools versions = all
23172025-08-26T21:47:03.069ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/silos/{silo}/policy versions = all
23182025-08-26T21:47:03.069ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = PUT path = /v1/system/silos/{silo}/policy versions = all
23192025-08-26T21:47:03.069ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/silos/{silo}/quotas versions = all
23202025-08-26T21:47:03.069ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = PUT path = /v1/system/silos/{silo}/quotas versions = all
23212025-08-26T21:47:03.069ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/system/timeseries/query versions = all
23222025-08-26T21:47:03.069ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/timeseries/schemas versions = all
23232025-08-26T21:47:03.069ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = PUT path = /v1/system/update/repository versions = all
23242025-08-26T21:47:03.069ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/update/repository/{system_version} versions = all
23252025-08-26T21:47:03.069ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/update/target-release versions = all
23262025-08-26T21:47:03.069ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = PUT path = /v1/system/update/target-release versions = all
23272025-08-26T21:47:03.069ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/update/trust-roots versions = all
23282025-08-26T21:47:03.069ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/system/update/trust-roots versions = all
23292025-08-26T21:47:03.069ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/system/update/trust-roots/{trust_root_id} versions = all
23302025-08-26T21:47:03.070ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/update/trust-roots/{trust_root_id} versions = all
23312025-08-26T21:47:03.070ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/users versions = all
23322025-08-26T21:47:03.070ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/users/{user_id} versions = all
23332025-08-26T21:47:03.070ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/users-builtin versions = all
23342025-08-26T21:47:03.070ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/users-builtin/{user} versions = all
23352025-08-26T21:47:03.070ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/utilization/silos versions = all
23362025-08-26T21:47:03.070ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/utilization/silos/{silo} versions = all
23372025-08-26T21:47:03.070ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/timeseries/query versions = all
23382025-08-26T21:47:03.070ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/users versions = all
23392025-08-26T21:47:03.070ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/users/{user_id} versions = all
23402025-08-26T21:47:03.070ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/users/{user_id}/access-tokens versions = all
23412025-08-26T21:47:03.070ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/users/{user_id}/logout versions = all
23422025-08-26T21:47:03.070ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/users/{user_id}/sessions versions = all
23432025-08-26T21:47:03.070ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/utilization versions = all
23442025-08-26T21:47:03.070ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/vpc-firewall-rules versions = all
23452025-08-26T21:47:03.070ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = PUT path = /v1/vpc-firewall-rules versions = all
23462025-08-26T21:47:03.070ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/vpc-router-routes versions = all
23472025-08-26T21:47:03.070ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/vpc-router-routes versions = all
23482025-08-26T21:47:03.070ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/vpc-router-routes/{route} versions = all
23492025-08-26T21:47:03.070ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/vpc-router-routes/{route} versions = all
23502025-08-26T21:47:03.070ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = PUT path = /v1/vpc-router-routes/{route} versions = all
23512025-08-26T21:47:03.070ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/vpc-routers versions = all
23522025-08-26T21:47:03.070ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/vpc-routers versions = all
23532025-08-26T21:47:03.070ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/vpc-routers/{router} versions = all
23542025-08-26T21:47:03.070ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/vpc-routers/{router} versions = all
23552025-08-26T21:47:03.070ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = PUT path = /v1/vpc-routers/{router} versions = all
23562025-08-26T21:47:03.070ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/vpc-subnets versions = all
23572025-08-26T21:47:03.070ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/vpc-subnets versions = all
23582025-08-26T21:47:03.070ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/vpc-subnets/{subnet} versions = all
23592025-08-26T21:47:03.070ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/vpc-subnets/{subnet} versions = all
23602025-08-26T21:47:03.070ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = PUT path = /v1/vpc-subnets/{subnet} versions = all
23612025-08-26T21:47:03.070ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/vpc-subnets/{subnet}/network-interfaces versions = all
23622025-08-26T21:47:03.070ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/vpcs versions = all
23632025-08-26T21:47:03.070ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/vpcs versions = all
23642025-08-26T21:47:03.070ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/vpcs/{vpc} versions = all
23652025-08-26T21:47:03.070ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/vpcs/{vpc} versions = all
23662025-08-26T21:47:03.070ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = PUT path = /v1/vpcs/{vpc} versions = all
23672025-08-26T21:47:03.070ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/webhook-receivers versions = all
23682025-08-26T21:47:03.070ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = PUT path = /v1/webhook-receivers/{receiver} versions = all
23692025-08-26T21:47:03.070ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/webhook-secrets versions = all
23702025-08-26T21:47:03.070ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/webhook-secrets versions = all
23712025-08-26T21:47:03.070ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/webhook-secrets/{secret_id} versions = all
23722025-08-26T21:47:03.070ZINFOnexus (dropshot_external): listening file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:328 local_addr = 172.30.2.7:443
23732025-08-26T21:47:03.070ZDEBGnexus (dropshot_external): successfully registered DTrace USDT probes local_addr = 172.30.2.7:443
23742025-08-26T21:47:03.073ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
23752025-08-26T21:47:03.074ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
23762025-08-26T21:47:03.076ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23772025-08-26T21:47:03.078ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
23782025-08-26T21:47:03.078ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
23792025-08-26T21:47:03.078ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
23802025-08-26T21:47:03.079ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = / versions = all
23812025-08-26T21:47:03.079ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /access versions = all
23822025-08-26T21:47:03.079ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /assets/{path} versions = all
23832025-08-26T21:47:03.079ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /device/auth versions = all
23842025-08-26T21:47:03.079ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /device/confirm versions = all
23852025-08-26T21:47:03.079ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /device/success versions = all
23862025-08-26T21:47:03.079ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /device/token versions = all
23872025-08-26T21:47:03.079ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /device/verify versions = all
23882025-08-26T21:47:03.079ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /experimental/v1/probes versions = all
23892025-08-26T21:47:03.079ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /experimental/v1/probes versions = all
23902025-08-26T21:47:03.079ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /experimental/v1/probes/{probe} versions = all
23912025-08-26T21:47:03.079ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /experimental/v1/probes/{probe} versions = all
23922025-08-26T21:47:03.079ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /experimental/v1/system/support-bundles versions = all
23932025-08-26T21:47:03.079ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /experimental/v1/system/support-bundles versions = all
23942025-08-26T21:47:03.079ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /experimental/v1/system/support-bundles/{bundle_id} versions = all
23952025-08-26T21:47:03.079ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /experimental/v1/system/support-bundles/{bundle_id} versions = all
23962025-08-26T21:47:03.079ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = PUT path = /experimental/v1/system/support-bundles/{bundle_id} versions = all
23972025-08-26T21:47:03.079ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /experimental/v1/system/support-bundles/{bundle_id}/download versions = all
23982025-08-26T21:47:03.079ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = HEAD path = /experimental/v1/system/support-bundles/{bundle_id}/download versions = all
23992025-08-26T21:47:03.079ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /experimental/v1/system/support-bundles/{bundle_id}/download/{file} versions = all
24002025-08-26T21:47:03.079ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = HEAD path = /experimental/v1/system/support-bundles/{bundle_id}/download/{file} versions = all
24012025-08-26T21:47:03.079ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /experimental/v1/system/support-bundles/{bundle_id}/index versions = all
24022025-08-26T21:47:03.079ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /images versions = all
24032025-08-26T21:47:03.079ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /login versions = all
24042025-08-26T21:47:03.079ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /login/{silo_name}/local versions = all
24052025-08-26T21:47:03.079ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /login/{silo_name}/saml/{provider_name} versions = all
24062025-08-26T21:47:03.079ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /login/{silo_name}/saml/{provider_name} versions = all
24072025-08-26T21:47:03.079ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /login/{silo_name}/saml/{provider_name}/redirect versions = all
24082025-08-26T21:47:03.079ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /lookup/{path} versions = all
24092025-08-26T21:47:03.079ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /projects/{path} versions = all
24102025-08-26T21:47:03.079ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /projects-new versions = all
24112025-08-26T21:47:03.079ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /settings/{path} versions = all
24122025-08-26T21:47:03.079ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /system/{path} versions = all
24132025-08-26T21:47:03.079ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /utilization versions = all
24142025-08-26T21:47:03.079ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/affinity-groups versions = all
24152025-08-26T21:47:03.079ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/affinity-groups versions = all
24162025-08-26T21:47:03.079ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/affinity-groups/{affinity_group} versions = all
24172025-08-26T21:47:03.079ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/affinity-groups/{affinity_group} versions = all
24182025-08-26T21:47:03.079ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = PUT path = /v1/affinity-groups/{affinity_group} versions = all
24192025-08-26T21:47:03.079ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/affinity-groups/{affinity_group}/members versions = all
24202025-08-26T21:47:03.079ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/affinity-groups/{affinity_group}/members/instance/{instance} versions = all
24212025-08-26T21:47:03.079ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/affinity-groups/{affinity_group}/members/instance/{instance} versions = all
24222025-08-26T21:47:03.079ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/affinity-groups/{affinity_group}/members/instance/{instance} versions = all
24232025-08-26T21:47:03.079ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/alert-classes versions = all
24242025-08-26T21:47:03.079ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/alert-receivers versions = all
24252025-08-26T21:47:03.079ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/alert-receivers/{receiver} versions = all
24262025-08-26T21:47:03.079ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/alert-receivers/{receiver} versions = all
24272025-08-26T21:47:03.079ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/alert-receivers/{receiver}/deliveries versions = all
24282025-08-26T21:47:03.079ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/alert-receivers/{receiver}/probe versions = all
24292025-08-26T21:47:03.079ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/alert-receivers/{receiver}/subscriptions versions = all
24302025-08-26T21:47:03.079ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/alert-receivers/{receiver}/subscriptions/{subscription} versions = all
24312025-08-26T21:47:03.079ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/alerts/{alert_id}/resend versions = all
24322025-08-26T21:47:03.079ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/anti-affinity-groups versions = all
24332025-08-26T21:47:03.079ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/anti-affinity-groups versions = all
24342025-08-26T21:47:03.079ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/anti-affinity-groups/{anti_affinity_group} versions = all
24352025-08-26T21:47:03.079ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/anti-affinity-groups/{anti_affinity_group} versions = all
24362025-08-26T21:47:03.079ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = PUT path = /v1/anti-affinity-groups/{anti_affinity_group} versions = all
24372025-08-26T21:47:03.079ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/anti-affinity-groups/{anti_affinity_group}/members versions = all
24382025-08-26T21:47:03.079ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/anti-affinity-groups/{anti_affinity_group}/members/instance/{instance} versions = all
24392025-08-26T21:47:03.079ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/anti-affinity-groups/{anti_affinity_group}/members/instance/{instance} versions = all
24402025-08-26T21:47:03.079ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/anti-affinity-groups/{anti_affinity_group}/members/instance/{instance} versions = all
24412025-08-26T21:47:03.079ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/auth-settings versions = all
24422025-08-26T21:47:03.079ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = PUT path = /v1/auth-settings versions = all
24432025-08-26T21:47:03.079ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/certificates versions = all
24442025-08-26T21:47:03.079ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/certificates versions = all
24452025-08-26T21:47:03.079ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/certificates/{certificate} versions = all
24462025-08-26T21:47:03.079ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/certificates/{certificate} versions = all
24472025-08-26T21:47:03.079ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/disks versions = all
24482025-08-26T21:47:03.079ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/disks versions = all
24492025-08-26T21:47:03.079ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/disks/{disk} versions = all
24502025-08-26T21:47:03.079ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/disks/{disk} versions = all
24512025-08-26T21:47:03.079ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/disks/{disk}/bulk-write versions = all
24522025-08-26T21:47:03.079ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/disks/{disk}/bulk-write-start versions = all
24532025-08-26T21:47:03.079ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/disks/{disk}/bulk-write-stop versions = all
24542025-08-26T21:47:03.079ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/disks/{disk}/finalize versions = all
24552025-08-26T21:47:03.079ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/floating-ips versions = all
24562025-08-26T21:47:03.079ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/floating-ips versions = all
24572025-08-26T21:47:03.079ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/floating-ips/{floating_ip} versions = all
24582025-08-26T21:47:03.080ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/floating-ips/{floating_ip} versions = all
24592025-08-26T21:47:03.080ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = PUT path = /v1/floating-ips/{floating_ip} versions = all
24602025-08-26T21:47:03.080ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/floating-ips/{floating_ip}/attach versions = all
24612025-08-26T21:47:03.080ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/floating-ips/{floating_ip}/detach versions = all
24622025-08-26T21:47:03.080ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/groups versions = all
24632025-08-26T21:47:03.080ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/groups/{group_id} versions = all
24642025-08-26T21:47:03.080ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/images versions = all
24652025-08-26T21:47:03.080ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/images versions = all
24662025-08-26T21:47:03.080ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/images/{image} versions = all
24672025-08-26T21:47:03.080ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/images/{image} versions = all
24682025-08-26T21:47:03.080ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/images/{image}/demote versions = all
24692025-08-26T21:47:03.080ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/images/{image}/promote versions = all
24702025-08-26T21:47:03.080ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/instances versions = all
24712025-08-26T21:47:03.080ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/instances versions = all
24722025-08-26T21:47:03.080ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/instances/{instance} versions = all
24732025-08-26T21:47:03.080ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/instances/{instance} versions = all
24742025-08-26T21:47:03.080ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = PUT path = /v1/instances/{instance} versions = all
24752025-08-26T21:47:03.080ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/instances/{instance}/affinity-groups versions = all
24762025-08-26T21:47:03.080ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/instances/{instance}/anti-affinity-groups versions = all
24772025-08-26T21:47:03.080ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/instances/{instance}/disks versions = all
24782025-08-26T21:47:03.080ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/instances/{instance}/disks/attach versions = all
24792025-08-26T21:47:03.080ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/instances/{instance}/disks/detach versions = all
24802025-08-26T21:47:03.080ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/instances/{instance}/external-ips versions = all
24812025-08-26T21:47:03.080ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/instances/{instance}/external-ips/ephemeral versions = all
24822025-08-26T21:47:03.080ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/instances/{instance}/external-ips/ephemeral versions = all
24832025-08-26T21:47:03.080ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/instances/{instance}/reboot versions = all
24842025-08-26T21:47:03.080ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/instances/{instance}/serial-console versions = all
24852025-08-26T21:47:03.080ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/instances/{instance}/serial-console/stream versions = all
24862025-08-26T21:47:03.080ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/instances/{instance}/ssh-public-keys versions = all
24872025-08-26T21:47:03.080ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/instances/{instance}/start versions = all
24882025-08-26T21:47:03.080ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/instances/{instance}/stop versions = all
24892025-08-26T21:47:03.080ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/internet-gateway-ip-addresses versions = all
24902025-08-26T21:47:03.080ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/internet-gateway-ip-addresses versions = all
24912025-08-26T21:47:03.080ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/internet-gateway-ip-addresses/{address} versions = all
24922025-08-26T21:47:03.080ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/internet-gateway-ip-pools versions = all
24932025-08-26T21:47:03.080ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/internet-gateway-ip-pools versions = all
24942025-08-26T21:47:03.080ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/internet-gateway-ip-pools/{pool} versions = all
24952025-08-26T21:47:03.080ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/internet-gateways versions = all
24962025-08-26T21:47:03.080ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/internet-gateways versions = all
24972025-08-26T21:47:03.080ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/internet-gateways/{gateway} versions = all
24982025-08-26T21:47:03.080ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/internet-gateways/{gateway} versions = all
24992025-08-26T21:47:03.080ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/ip-pools versions = all
25002025-08-26T21:47:03.080ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/ip-pools/{pool} versions = all
25012025-08-26T21:47:03.080ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/login/{silo_name}/local versions = all
25022025-08-26T21:47:03.080ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/logout versions = all
25032025-08-26T21:47:03.080ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/me versions = all
25042025-08-26T21:47:03.080ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/me/access-tokens versions = all
25052025-08-26T21:47:03.080ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/me/access-tokens/{token_id} versions = all
25062025-08-26T21:47:03.080ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/me/groups versions = all
25072025-08-26T21:47:03.080ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/me/ssh-keys versions = all
25082025-08-26T21:47:03.080ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/me/ssh-keys versions = all
25092025-08-26T21:47:03.080ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/me/ssh-keys/{ssh_key} versions = all
25102025-08-26T21:47:03.080ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/me/ssh-keys/{ssh_key} versions = all
25112025-08-26T21:47:03.080ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/metrics/{metric_name} versions = all
25122025-08-26T21:47:03.080ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/network-interfaces versions = all
25132025-08-26T21:47:03.080ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/network-interfaces versions = all
25142025-08-26T21:47:03.080ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/network-interfaces/{interface} versions = all
25152025-08-26T21:47:03.080ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/network-interfaces/{interface} versions = all
25162025-08-26T21:47:03.080ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = PUT path = /v1/network-interfaces/{interface} versions = all
25172025-08-26T21:47:03.080ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/ping versions = all
25182025-08-26T21:47:03.080ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/policy versions = all
25192025-08-26T21:47:03.080ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = PUT path = /v1/policy versions = all
25202025-08-26T21:47:03.080ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/projects versions = all
25212025-08-26T21:47:03.080ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/projects versions = all
25222025-08-26T21:47:03.080ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/projects/{project} versions = all
25232025-08-26T21:47:03.080ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/projects/{project} versions = all
25242025-08-26T21:47:03.080ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = PUT path = /v1/projects/{project} versions = all
25252025-08-26T21:47:03.080ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/projects/{project}/policy versions = all
25262025-08-26T21:47:03.080ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = PUT path = /v1/projects/{project}/policy versions = all
25272025-08-26T21:47:03.080ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/snapshots versions = all
25282025-08-26T21:47:03.080ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/snapshots versions = all
25292025-08-26T21:47:03.080ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/snapshots/{snapshot} versions = all
25302025-08-26T21:47:03.080ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/snapshots/{snapshot} versions = all
25312025-08-26T21:47:03.080ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/audit-log versions = all
25322025-08-26T21:47:03.080ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/hardware/disks versions = all
25332025-08-26T21:47:03.080ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/hardware/disks/{disk_id} versions = all
25342025-08-26T21:47:03.080ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/hardware/rack-switch-port/{rack_id}/{switch_location}/{port}/lldp/neighbors versions = all
25352025-08-26T21:47:03.080ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/hardware/racks versions = all
25362025-08-26T21:47:03.080ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/hardware/racks/{rack_id} versions = all
25372025-08-26T21:47:03.080ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/hardware/sleds versions = all
25382025-08-26T21:47:03.080ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/system/hardware/sleds versions = all
25392025-08-26T21:47:03.080ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/hardware/sleds/{sled_id} versions = all
25402025-08-26T21:47:03.080ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/hardware/sleds/{sled_id}/disks versions = all
25412025-08-26T21:47:03.080ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/hardware/sleds/{sled_id}/instances versions = all
25422025-08-26T21:47:03.080ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = PUT path = /v1/system/hardware/sleds/{sled_id}/provision-policy versions = all
25432025-08-26T21:47:03.080ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/hardware/sleds-uninitialized versions = all
25442025-08-26T21:47:03.080ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/hardware/switch-port versions = all
25452025-08-26T21:47:03.080ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/hardware/switch-port/{port}/lldp/config versions = all
25462025-08-26T21:47:03.080ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/system/hardware/switch-port/{port}/lldp/config versions = all
25472025-08-26T21:47:03.080ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/system/hardware/switch-port/{port}/settings versions = all
25482025-08-26T21:47:03.080ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/system/hardware/switch-port/{port}/settings versions = all
25492025-08-26T21:47:03.080ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/hardware/switch-port/{port}/status versions = all
25502025-08-26T21:47:03.080ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/hardware/switches versions = all
25512025-08-26T21:47:03.080ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/hardware/switches/{switch_id} versions = all
25522025-08-26T21:47:03.080ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/identity-providers versions = all
25532025-08-26T21:47:03.082ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/system/identity-providers/local/users versions = all
25542025-08-26T21:47:03.082ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/system/identity-providers/local/users/{user_id} versions = all
25552025-08-26T21:47:03.082ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/system/identity-providers/local/users/{user_id}/set-password versions = all
25562025-08-26T21:47:03.082ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/system/identity-providers/saml versions = all
25572025-08-26T21:47:03.082ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/identity-providers/saml/{provider} versions = all
25582025-08-26T21:47:03.082ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/ip-pools versions = all
25592025-08-26T21:47:03.082ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/system/ip-pools versions = all
25602025-08-26T21:47:03.082ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/system/ip-pools/{pool} versions = all
25612025-08-26T21:47:03.082ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/ip-pools/{pool} versions = all
25622025-08-26T21:47:03.082ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = PUT path = /v1/system/ip-pools/{pool} versions = all
25632025-08-26T21:47:03.082ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/ip-pools/{pool}/ranges versions = all
25642025-08-26T21:47:03.082ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/system/ip-pools/{pool}/ranges/add versions = all
25652025-08-26T21:47:03.083ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/system/ip-pools/{pool}/ranges/remove versions = all
25662025-08-26T21:47:03.083ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/ip-pools/{pool}/silos versions = all
25672025-08-26T21:47:03.083ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/system/ip-pools/{pool}/silos versions = all
25682025-08-26T21:47:03.083ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/system/ip-pools/{pool}/silos/{silo} versions = all
25692025-08-26T21:47:03.083ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = PUT path = /v1/system/ip-pools/{pool}/silos/{silo} versions = all
25702025-08-26T21:47:03.083ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/ip-pools/{pool}/utilization versions = all
25712025-08-26T21:47:03.083ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/ip-pools-service versions = all
25722025-08-26T21:47:03.083ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/ip-pools-service/ranges versions = all
25732025-08-26T21:47:03.083ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/system/ip-pools-service/ranges/add versions = all
25742025-08-26T21:47:03.083ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/system/ip-pools-service/ranges/remove versions = all
25752025-08-26T21:47:03.083ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/metrics/{metric_name} versions = all
25762025-08-26T21:47:03.083ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/networking/address-lot versions = all
25772025-08-26T21:47:03.083ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/system/networking/address-lot versions = all
25782025-08-26T21:47:03.083ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/system/networking/address-lot/{address_lot} versions = all
25792025-08-26T21:47:03.083ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/networking/address-lot/{address_lot} versions = all
25802025-08-26T21:47:03.083ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/networking/address-lot/{address_lot}/blocks versions = all
25812025-08-26T21:47:03.083ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/networking/allow-list versions = all
25822025-08-26T21:47:03.083ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = PUT path = /v1/system/networking/allow-list versions = all
25832025-08-26T21:47:03.083ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/system/networking/bfd-disable versions = all
25842025-08-26T21:47:03.083ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/system/networking/bfd-enable versions = all
25852025-08-26T21:47:03.083ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/networking/bfd-status versions = all
25862025-08-26T21:47:03.083ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/system/networking/bgp versions = all
25872025-08-26T21:47:03.083ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/networking/bgp versions = all
25882025-08-26T21:47:03.083ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/system/networking/bgp versions = all
25892025-08-26T21:47:03.083ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/networking/bgp-announce-set versions = all
25902025-08-26T21:47:03.083ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = PUT path = /v1/system/networking/bgp-announce-set versions = all
25912025-08-26T21:47:03.083ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/system/networking/bgp-announce-set/{announce_set} versions = all
25922025-08-26T21:47:03.083ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/networking/bgp-announce-set/{announce_set}/announcement versions = all
25932025-08-26T21:47:03.083ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/networking/bgp-exported versions = all
25942025-08-26T21:47:03.083ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/networking/bgp-message-history versions = all
25952025-08-26T21:47:03.083ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/networking/bgp-routes-ipv4 versions = all
25962025-08-26T21:47:03.083ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/networking/bgp-status versions = all
25972025-08-26T21:47:03.083ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/networking/inbound-icmp versions = all
25982025-08-26T21:47:03.083ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = PUT path = /v1/system/networking/inbound-icmp versions = all
25992025-08-26T21:47:03.083ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/networking/loopback-address versions = all
26002025-08-26T21:47:03.083ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/system/networking/loopback-address versions = all
26012025-08-26T21:47:03.083ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/system/networking/loopback-address/{rack_id}/{switch_location}/{address}/{subnet_mask} versions = all
26022025-08-26T21:47:03.083ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/system/networking/switch-port-settings versions = all
26032025-08-26T21:47:03.083ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/networking/switch-port-settings versions = all
26042025-08-26T21:47:03.083ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/system/networking/switch-port-settings versions = all
26052025-08-26T21:47:03.083ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/networking/switch-port-settings/{port} versions = all
26062025-08-26T21:47:03.083ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/policy versions = all
26072025-08-26T21:47:03.083ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = PUT path = /v1/system/policy versions = all
26082025-08-26T21:47:03.083ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/silo-quotas versions = all
26092025-08-26T21:47:03.083ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/silos versions = all
26102025-08-26T21:47:03.083ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/system/silos versions = all
26112025-08-26T21:47:03.083ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/system/silos/{silo} versions = all
26122025-08-26T21:47:03.083ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/silos/{silo} versions = all
26132025-08-26T21:47:03.083ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/silos/{silo}/ip-pools versions = all
26142025-08-26T21:47:03.083ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/silos/{silo}/policy versions = all
26152025-08-26T21:47:03.083ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = PUT path = /v1/system/silos/{silo}/policy versions = all
26162025-08-26T21:47:03.083ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/silos/{silo}/quotas versions = all
26172025-08-26T21:47:03.083ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = PUT path = /v1/system/silos/{silo}/quotas versions = all
26182025-08-26T21:47:03.083ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/system/timeseries/query versions = all
26192025-08-26T21:47:03.083ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/timeseries/schemas versions = all
26202025-08-26T21:47:03.083ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = PUT path = /v1/system/update/repository versions = all
26212025-08-26T21:47:03.083ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/update/repository/{system_version} versions = all
26222025-08-26T21:47:03.083ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/update/target-release versions = all
26232025-08-26T21:47:03.083ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = PUT path = /v1/system/update/target-release versions = all
26242025-08-26T21:47:03.083ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/update/trust-roots versions = all
26252025-08-26T21:47:03.083ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/system/update/trust-roots versions = all
26262025-08-26T21:47:03.083ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/system/update/trust-roots/{trust_root_id} versions = all
26272025-08-26T21:47:03.083ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/update/trust-roots/{trust_root_id} versions = all
26282025-08-26T21:47:03.083ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/users versions = all
26292025-08-26T21:47:03.083ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/users/{user_id} versions = all
26302025-08-26T21:47:03.083ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/users-builtin versions = all
26312025-08-26T21:47:03.083ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/users-builtin/{user} versions = all
26322025-08-26T21:47:03.083ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/utilization/silos versions = all
26332025-08-26T21:47:03.083ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/utilization/silos/{silo} versions = all
26342025-08-26T21:47:03.083ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/timeseries/query versions = all
26352025-08-26T21:47:03.083ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/users versions = all
26362025-08-26T21:47:03.083ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/users/{user_id} versions = all
26372025-08-26T21:47:03.083ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/users/{user_id}/access-tokens versions = all
26382025-08-26T21:47:03.083ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/users/{user_id}/logout versions = all
26392025-08-26T21:47:03.083ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/users/{user_id}/sessions versions = all
26402025-08-26T21:47:03.083ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/utilization versions = all
26412025-08-26T21:47:03.083ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/vpc-firewall-rules versions = all
26422025-08-26T21:47:03.083ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = PUT path = /v1/vpc-firewall-rules versions = all
26432025-08-26T21:47:03.083ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/vpc-router-routes versions = all
26442025-08-26T21:47:03.083ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/vpc-router-routes versions = all
26452025-08-26T21:47:03.083ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/vpc-router-routes/{route} versions = all
26462025-08-26T21:47:03.083ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/vpc-router-routes/{route} versions = all
26472025-08-26T21:47:03.083ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = PUT path = /v1/vpc-router-routes/{route} versions = all
26482025-08-26T21:47:03.083ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/vpc-routers versions = all
26492025-08-26T21:47:03.083ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/vpc-routers versions = all
26502025-08-26T21:47:03.083ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/vpc-routers/{router} versions = all
26512025-08-26T21:47:03.083ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/vpc-routers/{router} versions = all
26522025-08-26T21:47:03.083ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = PUT path = /v1/vpc-routers/{router} versions = all
26532025-08-26T21:47:03.083ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/vpc-subnets versions = all
26542025-08-26T21:47:03.083ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/vpc-subnets versions = all
26552025-08-26T21:47:03.083ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/vpc-subnets/{subnet} versions = all
26562025-08-26T21:47:03.083ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/vpc-subnets/{subnet} versions = all
26572025-08-26T21:47:03.083ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = PUT path = /v1/vpc-subnets/{subnet} versions = all
26582025-08-26T21:47:03.083ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/vpc-subnets/{subnet}/network-interfaces versions = all
26592025-08-26T21:47:03.083ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/vpcs versions = all
26602025-08-26T21:47:03.083ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/vpcs versions = all
26612025-08-26T21:47:03.083ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/vpcs/{vpc} versions = all
26622025-08-26T21:47:03.083ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/vpcs/{vpc} versions = all
26632025-08-26T21:47:03.083ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = PUT path = /v1/vpcs/{vpc} versions = all
26642025-08-26T21:47:03.083ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/webhook-receivers versions = all
26652025-08-26T21:47:03.083ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = PUT path = /v1/webhook-receivers/{receiver} versions = all
26662025-08-26T21:47:03.083ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/webhook-secrets versions = all
26672025-08-26T21:47:03.083ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/webhook-secrets versions = all
26682025-08-26T21:47:03.083ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/webhook-secrets/{secret_id} versions = all
26692025-08-26T21:47:03.084ZINFOnexus (dropshot_external_techport): listening file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:328 local_addr = [fd00:1122:3344:101::c]:12228
26702025-08-26T21:47:03.084ZDEBGnexus (dropshot_external_techport): successfully registered DTrace USDT probes local_addr = [fd00:1122:3344:101::c]:12228
26712025-08-26T21:47:03.084ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26722025-08-26T21:47:03.084ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
26732025-08-26T21:47:03.084ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
26742025-08-26T21:47:03.084ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
26752025-08-26T21:47:03.084ZDEBGnexus (nexus-producer-server): registered DTrace probes
26762025-08-26T21:47:03.091ZDEBGnexus (dropshot): registered endpoint local_addr = [fd00:1122:3344:101::c]:42016 method = GET path = /{producer_id} versions = all
26772025-08-26T21:47:03.091ZINFOnexus (dropshot): listening file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:328 local_addr = [fd00:1122:3344:101::c]:42016
26782025-08-26T21:47:03.091ZDEBGnexus (dropshot): successfully registered DTrace USDT probes local_addr = [fd00:1122:3344:101::c]:42016
26792025-08-26T21:47:03.091ZDEBGnexus (nexus-producer-server): Requested any available port, Dropshot server has been bound to [fd00:1122:3344:101::c]:42016
26802025-08-26T21:47:03.091ZDEBGnexus (nexus-producer-server): Nexus IP provided explicitly, will use it addr = [fd00:1122:3344:101::c]:12221
26812025-08-26T21:47:03.091ZDEBGnexus (nexus-producer-server): starting producer registration task
26822025-08-26T21:47:03.092ZINFOnexus (nexus-producer-server): starting oximeter metric producer server address = [fd00:1122:3344:101::c]:42016 file = oximeter/producer/src/lib.rs:279 interval = 10s producer_id = 5d0959e5-47a6-4bf3-a52b-57950b9d900d
26832025-08-26T21:47:03.092ZDEBGnexus (producer-registration-task): registering as oximeter producer with Nexus registration_count = 0
26842025-08-26T21:47:03.092ZDEBGnexus (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::c]:12221
26852025-08-26T21:47:03.098ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
26862025-08-26T21:47:03.099ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
26872025-08-26T21:47:03.100ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 265.568066ms iteration = 3 status = null
26882025-08-26T21:47:03.125ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
26892025-08-26T21:47:03.143ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 308.347041ms iteration = 3 status = {}
26902025-08-26T21:47:03.178ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
26912025-08-26T21:47:03.178ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
26922025-08-26T21:47:03.179ZINFOnexus (BackgroundTasks): instance reincarnation completed successfully background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:113 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0
26932025-08-26T21:47:03.179ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 317.325434ms iteration = 2 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
26942025-08-26T21:47:03.180ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
26952025-08-26T21:47:03.181ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
26962025-08-26T21:47:03.182ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26972025-08-26T21:47:03.185ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: dd0c655a-95c2-48fc-b4c3-9733ed4b0faa, lookup_type: ById(dd0c655a-95c2-48fc-b4c3-9733ed4b0faa) } result = Ok(())
26982025-08-26T21:47:03.185ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
26992025-08-26T21:47:03.185ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
27002025-08-26T21:47:03.187ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 325.460796ms iteration = 3 status = 0
27012025-08-26T21:47:03.191ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1213 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 zone_address = fd00:1122:3344:101::2
27022025-08-26T21:47:03.191ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
27032025-08-26T21:47:03.191ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "130a1a4d-c3ff-4fa8-8145-74fad5d77766", "content-length": "26", "date": "Tue, 26 Aug 2025 21:47:02 GMT"} })
27042025-08-26T21:47:03.191ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1216 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
27052025-08-26T21:47:03.193ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
27062025-08-26T21:47:03.268ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
27072025-08-26T21:47:03.269ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "ab1c9b44-c6a0-413c-951d-655023fc187d", "content-length": "691", "date": "Tue, 26 Aug 2025 21:47:03 GMT"} })
27082025-08-26T21:47:03.269ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
27092025-08-26T21:47:03.269ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
27102025-08-26T21:47:03.269ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
27112025-08-26T21:47:03.271ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 409.00435ms iteration = 3 status = {}
27122025-08-26T21:47:03.278ZDEBGnexus (producer-registration-task): client request body = Some(Body) method = POST uri = http://[fd00:1122:3344:101::c]:12221/metrics/producers
27132025-08-26T21:47:03.278ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:1025 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::c]:47718
27142025-08-26T21:47:03.287ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:47718 req_id = 81825137-c8d7-441a-8d92-c788dd77afae roles = RoleSet { roles: {} } uri = /metrics/producers
27152025-08-26T21:47:03.288ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:47718 req_id = 81825137-c8d7-441a-8d92-c788dd77afae resource = Database result = Ok(()) uri = /metrics/producers
27162025-08-26T21:47:03.294ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
27172025-08-26T21:47:03.294ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "d8921d1e-de3b-44d4-9a44-29e66ca599eb", "date": "Tue, 26 Aug 2025 21:47:03 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
27182025-08-26T21:47:03.294ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
27192025-08-26T21:47:03.296ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "9b6dab18-1983-4057-b063-9e29200a430a", "date": "Tue, 26 Aug 2025 21:47:03 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
27202025-08-26T21:47:03.296ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
27212025-08-26T21:47:03.297ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
27222025-08-26T21:47:03.297ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 364.517257ms iteration = 4 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
27232025-08-26T21:47:03.316ZINFOnexus (BackgroundTasks): VPC route manager sled dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
27242025-08-26T21:47:03.316ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
27252025-08-26T21:47:03.317ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "611f4954-93bf-4609-abe7-459b629c519e", "content-length": "372", "date": "Tue, 26 Aug 2025 21:47:02 GMT"} })
27262025-08-26T21:47:03.317ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
27272025-08-26T21:47:03.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
27282025-08-26T21:47:03.317ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
27292025-08-26T21:47:03.331ZINFOnexus: assigned collector to new producer collector_id = f843ff0e-eef8-49c4-b428-23e15df1def3 file = nexus/src/app/oximeter.rs:76 producer_id = 5d0959e5-47a6-4bf3-a52b-57950b9d900d
27302025-08-26T21:47:03.331ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 53163 local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:47718 req_id = 81825137-c8d7-441a-8d92-c788dd77afae response_code = 201 uri = /metrics/producers
27312025-08-26T21:47:03.331ZDEBGnexus (producer-registration-task): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::c]:12221/metrics/producers", status: 201, headers: {"content-type": "application/json", "x-request-id": "81825137-c8d7-441a-8d92-c788dd77afae", "content-length": "41", "date": "Tue, 26 Aug 2025 21:47:03 GMT"} })
27322025-08-26T21:47:03.333ZDEBGnexus (producer-registration-task): successfully registered with Nexus, pausing until time to renew lease lease_duration = 600s wait_period = 150s
27332025-08-26T21:47:03.340ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
27342025-08-26T21:47:03.340ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
27352025-08-26T21:47:03.340ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "0a70c87b-4443-43f3-ae6c-ddd0640b5e59", "date": "Tue, 26 Aug 2025 21:47:03 GMT"} })
27362025-08-26T21:47:03.340ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
27372025-08-26T21:47:03.341ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
27382025-08-26T21:47:03.343ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
27392025-08-26T21:47:03.343ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
27402025-08-26T21:47:03.346ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
27412025-08-26T21:47:03.346ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
27422025-08-26T21:47:03.348ZINFOnexus (BackgroundTasks): VPC route manager sled dd0c655a-95c2-48fc-b4c3-9733ed4b0faa push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
27432025-08-26T21:47:03.348ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
27442025-08-26T21:47:03.349ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "9daa9020-9949-4a6a-b755-a19038865051", "date": "Tue, 26 Aug 2025 21:47:02 GMT"} })
27452025-08-26T21:47:03.351ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 489.012613ms iteration = 3 status = {}
27462025-08-26T21:47:03.376ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
27472025-08-26T21:47:03.438ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
27482025-08-26T21:47:03.438ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "49a56425-cc75-47eb-8665-e0c2467f4b4c", "date": "Tue, 26 Aug 2025 21:47:03 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
27492025-08-26T21:47:03.438ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
27502025-08-26T21:47:03.439ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "a975f0d7-9d7c-49dd-b3a8-50ac7eb2b3fe", "date": "Tue, 26 Aug 2025 21:47:03 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
27512025-08-26T21:47:03.439ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
27522025-08-26T21:47:03.441ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "ec139ba4-253a-4c6b-8f3d-2d26f3b6da9d", "date": "Tue, 26 Aug 2025 21:47:03 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
27532025-08-26T21:47:03.441ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
27542025-08-26T21:47:03.442ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
27552025-08-26T21:47:03.442ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
27562025-08-26T21:47:03.442ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
27572025-08-26T21:47:03.442ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 453.326549ms iteration = 4 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
27582025-08-26T21:47:03.444ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: d24c5c55-3a21-4a91-aeb4-45362e822df6, rack_id: e37a46f2-f789-4ca4-9841-f8c6dd705964, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(88d84f37-a4c8-4abc-af1f-27987b0d39d9) } rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0
27592025-08-26T21:47:03.444ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
27602025-08-26T21:47:03.444ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
27612025-08-26T21:47:03.457ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1832 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
27622025-08-26T21:47:03.457ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1859 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 88d84f37-a4c8-4abc-af1f-27987b0d39d9
27632025-08-26T21:47:03.457ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
27642025-08-26T21:47:03.457ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
27652025-08-26T21:47:03.457ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
27662025-08-26T21:47:03.457ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
27672025-08-26T21:47:03.457ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
27682025-08-26T21:47:03.457ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2006 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
27692025-08-26T21:47:03.457ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:408 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
27702025-08-26T21:47:03.457ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2042 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
27712025-08-26T21:47:03.457ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
27722025-08-26T21:47:03.457ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
27732025-08-26T21:47:03.458ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
27742025-08-26T21:47:03.460ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0
27752025-08-26T21:47:03.460ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
27762025-08-26T21:47:03.566ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "c1d5ee30-f94c-45e3-9b16-987a4f6af04c", "date": "Tue, 26 Aug 2025 21:47:03 GMT"} })
27772025-08-26T21:47:03.566ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch = Switch0
27782025-08-26T21:47:03.566ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
27792025-08-26T21:47:03.566ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "c44fab80-4871-4c2d-a719-7f7a831563aa", "content-length": "682", "date": "Tue, 26 Aug 2025 21:47:03 GMT"} })
27802025-08-26T21:47:03.566ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
27812025-08-26T21:47:03.567ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
27822025-08-26T21:47:03.570ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
27832025-08-26T21:47:03.571ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
27842025-08-26T21:47:03.574ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
27852025-08-26T21:47:03.575ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
27862025-08-26T21:47:03.576ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
27872025-08-26T21:47:03.576ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
27882025-08-26T21:47:03.584ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 721.934414ms iteration = 3 status = {}
27892025-08-26T21:47:07.848ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 14 reason = Timeout
27902025-08-26T21:47:07.848ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
27912025-08-26T21:47:07.848ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
27922025-08-26T21:47:07.850ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27932025-08-26T21:47:07.852ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
27942025-08-26T21:47:07.852ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
27952025-08-26T21:47:07.852ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
27962025-08-26T21:47:07.854ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
27972025-08-26T21:47:07.854ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 6.713701ms iteration = 14 status = {"chicken_switches_updated":false}
27982025-08-26T21:47:12.847ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 15 reason = Timeout
27992025-08-26T21:47:12.847ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
28002025-08-26T21:47:12.848ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
28012025-08-26T21:47:12.849ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28022025-08-26T21:47:12.852ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
28032025-08-26T21:47:12.852ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
28042025-08-26T21:47:12.852ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
28052025-08-26T21:47:12.853ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
28062025-08-26T21:47:12.853ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 6.270611ms iteration = 15 status = {"chicken_switches_updated":false}
28072025-08-26T21:47:12.859ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 8 reason = Timeout
28082025-08-26T21:47:12.859ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
28092025-08-26T21:47:12.860ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
28102025-08-26T21:47:12.862ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
28112025-08-26T21:47:12.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
28122025-08-26T21:47:12.864ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.42824ms iteration = 8 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
28132025-08-26T21:47:12.872ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 8 reason = Timeout
28142025-08-26T21:47:12.872ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
28152025-08-26T21:47:12.872ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
28162025-08-26T21:47:12.874ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28172025-08-26T21:47:12.877ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
28182025-08-26T21:47:12.877ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
28192025-08-26T21:47:12.877ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
28202025-08-26T21:47:12.879ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
28212025-08-26T21:47:12.880ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
28222025-08-26T21:47:12.881ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28232025-08-26T21:47:12.883ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
28242025-08-26T21:47:12.883ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
28252025-08-26T21:47:12.884ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
28262025-08-26T21:47:12.897ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
28272025-08-26T21:47:12.897ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 24.785789ms iteration = 8 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
28282025-08-26T21:47:15.656ZINFOnexus (dropshot): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:1025 local_addr = [fd00:1122:3344:101::c]:42016 remote_addr = [fd00:1122:3344:101::d]:37550
28292025-08-26T21:47:15.661ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 4381 local_addr = [fd00:1122:3344:101::c]:42016 method = GET remote_addr = [fd00:1122:3344:101::d]:37550 req_id = 51d75b58-2112-4c2f-8659-4c4d347469f3 response_code = 200 uri = /5d0959e5-47a6-4bf3-a52b-57950b9d900d
28302025-08-26T21:47:17.847ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 16 reason = Timeout
28312025-08-26T21:47:17.847ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
28322025-08-26T21:47:17.847ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
28332025-08-26T21:47:17.849ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28342025-08-26T21:47:17.852ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
28352025-08-26T21:47:17.852ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
28362025-08-26T21:47:17.853ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
28372025-08-26T21:47:17.855ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
28382025-08-26T21:47:17.855ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.727164ms iteration = 16 status = {"chicken_switches_updated":false}
28392025-08-26T21:47:22.848ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 17 reason = Timeout
28402025-08-26T21:47:22.848ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
28412025-08-26T21:47:22.849ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
28422025-08-26T21:47:22.850ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28432025-08-26T21:47:22.853ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
28442025-08-26T21:47:22.853ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
28452025-08-26T21:47:22.853ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
28462025-08-26T21:47:22.856ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
28472025-08-26T21:47:22.856ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.486608ms iteration = 17 status = {"chicken_switches_updated":false}
28482025-08-26T21:47:22.861ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 9 reason = Timeout
28492025-08-26T21:47:22.861ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
28502025-08-26T21:47:22.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
28512025-08-26T21:47:22.864ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
28522025-08-26T21:47:22.864ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
28532025-08-26T21:47:22.866ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.4154ms iteration = 9 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
28542025-08-26T21:47:22.873ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 9 reason = Timeout
28552025-08-26T21:47:22.873ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
28562025-08-26T21:47:22.873ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
28572025-08-26T21:47:22.875ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28582025-08-26T21:47:22.878ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
28592025-08-26T21:47:22.878ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
28602025-08-26T21:47:22.878ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
28612025-08-26T21:47:22.881ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
28622025-08-26T21:47:22.881ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
28632025-08-26T21:47:22.882ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28642025-08-26T21:47:22.885ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
28652025-08-26T21:47:22.885ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
28662025-08-26T21:47:22.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
28672025-08-26T21:47:22.899ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
28682025-08-26T21:47:22.899ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 26.329314ms iteration = 9 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
28692025-08-26T21:47:25.353ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 312 local_addr = [fd00:1122:3344:101::c]:42016 method = GET remote_addr = [fd00:1122:3344:101::d]:37550 req_id = d7585162-534e-40d3-9653-e233417c0a23 response_code = 200 uri = /5d0959e5-47a6-4bf3-a52b-57950b9d900d
28702025-08-26T21:47:27.848ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 18 reason = Timeout
28712025-08-26T21:47:27.848ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
28722025-08-26T21:47:27.849ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
28732025-08-26T21:47:27.850ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28742025-08-26T21:47:27.853ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
28752025-08-26T21:47:27.853ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
28762025-08-26T21:47:27.853ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
28772025-08-26T21:47:27.855ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
28782025-08-26T21:47:27.855ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 6.368293ms iteration = 18 status = {"chicken_switches_updated":false}
28792025-08-26T21:47:30.346ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:1025 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::d]:60237
28802025-08-26T21:47:30.346ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60237 req_id = c02a59e7-1477-4294-a118-7d39e4d3f741 roles = RoleSet { roles: {} } uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?sort_by=id_ascending
28812025-08-26T21:47:30.346ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60237 req_id = c02a59e7-1477-4294-a118-7d39e4d3f741 resource = Database result = Ok(()) uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?sort_by=id_ascending
28822025-08-26T21:47:30.348ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 2667 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60237 req_id = c02a59e7-1477-4294-a118-7d39e4d3f741 response_code = 200 uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?sort_by=id_ascending
28832025-08-26T21:47:30.349ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60237 req_id = c0e47f40-1972-4946-8c1b-6d44d4a21d17 roles = RoleSet { roles: {} } uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMzhjYTg4YS1lZGE1LTQwZGEtOGE4MS1hN2UzMzdhMmM4ZmIifX0%3D
28842025-08-26T21:47:30.349ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60237 req_id = c0e47f40-1972-4946-8c1b-6d44d4a21d17 resource = Database result = Ok(()) uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMzhjYTg4YS1lZGE1LTQwZGEtOGE4MS1hN2UzMzdhMmM4ZmIifX0%3D
28852025-08-26T21:47:30.351ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 2379 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60237 req_id = c0e47f40-1972-4946-8c1b-6d44d4a21d17 response_code = 200 uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMzhjYTg4YS1lZGE1LTQwZGEtOGE4MS1hN2UzMzdhMmM4ZmIifX0%3D
28862025-08-26T21:47:32.834ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 4 reason = Timeout
28872025-08-26T21:47:32.834ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 4 reason = Timeout
28882025-08-26T21:47:32.834ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
28892025-08-26T21:47:32.834ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 4 reason = Timeout
28902025-08-26T21:47:32.834ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
28912025-08-26T21:47:32.834ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
28922025-08-26T21:47:32.834ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
28932025-08-26T21:47:32.837ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 956, tv_nsec: 695273768 } })
28942025-08-26T21:47:32.838ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 956, tv_nsec: 695383571 } })
28952025-08-26T21:47:32.838ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1205
28962025-08-26T21:47:32.839ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 4.679686ms iteration = 4 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
28972025-08-26T21:47:32.839ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1205
28982025-08-26T21:47:32.848ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 19 reason = Timeout
28992025-08-26T21:47:32.848ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
29002025-08-26T21:47:32.848ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
29012025-08-26T21:47:32.850ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29022025-08-26T21:47:32.853ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
29032025-08-26T21:47:32.853ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
29042025-08-26T21:47:32.853ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
29052025-08-26T21:47:32.856ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
29062025-08-26T21:47:32.856ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.518259ms iteration = 19 status = {"chicken_switches_updated":false}
29072025-08-26T21:47:32.861ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 4 reason = Timeout
29082025-08-26T21:47:32.861ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
29092025-08-26T21:47:32.861ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 4 reason = Timeout
29102025-08-26T21:47:32.861ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 4 reason = Timeout
29112025-08-26T21:47:32.861ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
29122025-08-26T21:47:32.861ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
29132025-08-26T21:47:32.861ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 4 reason = Timeout
29142025-08-26T21:47:32.861ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
29152025-08-26T21:47:32.861ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
29162025-08-26T21:47:32.861ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 4 reason = Timeout
29172025-08-26T21:47:32.861ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
29182025-08-26T21:47:32.861ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
29192025-08-26T21:47:32.861ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 4 reason = Timeout
29202025-08-26T21:47:32.861ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
29212025-08-26T21:47:32.861ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 4 reason = Timeout
29222025-08-26T21:47:32.861ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
29232025-08-26T21:47:32.861ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 4 reason = Timeout
29242025-08-26T21:47:32.861ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 10 reason = Timeout
29252025-08-26T21:47:32.861ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
29262025-08-26T21:47:32.861ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 5 reason = Timeout
29272025-08-26T21:47:32.861ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
29282025-08-26T21:47:32.861ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
29292025-08-26T21:47:32.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
29302025-08-26T21:47:32.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
29312025-08-26T21:47:32.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
29322025-08-26T21:47:32.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
29332025-08-26T21:47:32.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
29342025-08-26T21:47:32.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
29352025-08-26T21:47:32.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
29362025-08-26T21:47:32.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
29372025-08-26T21:47:32.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
29382025-08-26T21:47:32.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
29392025-08-26T21:47:32.862ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 4 reason = Timeout
29402025-08-26T21:47:32.862ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 4 reason = Timeout
29412025-08-26T21:47:32.862ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
29422025-08-26T21:47:32.862ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 21.4\u{b5}s iteration = 4 status = {"disabled":true,"errors":[],"sps":[]}
29432025-08-26T21:47:32.862ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 4 reason = Timeout
29442025-08-26T21:47:32.862ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 4 reason = Timeout
29452025-08-26T21:47:32.862ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
29462025-08-26T21:47:32.862ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
29472025-08-26T21:47:32.862ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 4 reason = Timeout
29482025-08-26T21:47:32.862ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
29492025-08-26T21:47:32.862ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 4 reason = Timeout
29502025-08-26T21:47:32.862ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
29512025-08-26T21:47:32.863ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
29522025-08-26T21:47:32.863ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
29532025-08-26T21:47:32.863ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
29542025-08-26T21:47:32.863ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
29552025-08-26T21:47:32.863ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
29562025-08-26T21:47:32.863ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
29572025-08-26T21:47:32.863ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29582025-08-26T21:47:32.863ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
29592025-08-26T21:47:32.863ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29602025-08-26T21:47:32.864ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29612025-08-26T21:47:32.864ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29622025-08-26T21:47:32.864ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29632025-08-26T21:47:32.864ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29642025-08-26T21:47:32.865ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
29652025-08-26T21:47:32.865ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
29662025-08-26T21:47:32.865ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
29672025-08-26T21:47:32.865ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 2.539587ms iteration = 4 status = {"errors":[],"garbage_collect_requested":[]}
29682025-08-26T21:47:32.865ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
29692025-08-26T21:47:32.868ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
29702025-08-26T21:47:32.868ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
29712025-08-26T21:47:32.868ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
29722025-08-26T21:47:32.868ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
29732025-08-26T21:47:32.868ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
29742025-08-26T21:47:32.868ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
29752025-08-26T21:47:32.868ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
29762025-08-26T21:47:32.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
29772025-08-26T21:47:32.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
29782025-08-26T21:47:32.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
29792025-08-26T21:47:32.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
29802025-08-26T21:47:32.868ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
29812025-08-26T21:47:32.868ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
29822025-08-26T21:47:32.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
29832025-08-26T21:47:32.869ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
29842025-08-26T21:47:32.869ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
29852025-08-26T21:47:32.869ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
29862025-08-26T21:47:32.869ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
29872025-08-26T21:47:32.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
29882025-08-26T21:47:32.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
29892025-08-26T21:47:32.870ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 9.201358ms iteration = 4 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
29902025-08-26T21:47:32.870ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29912025-08-26T21:47:32.870ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
29922025-08-26T21:47:32.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
29932025-08-26T21:47:32.871ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 956, tv_nsec: 729159862 } })
29942025-08-26T21:47:32.872ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
29952025-08-26T21:47:32.872ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1205 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
29962025-08-26T21:47:32.872ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
29972025-08-26T21:47:32.872ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29982025-08-26T21:47:32.873ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
29992025-08-26T21:47:32.873ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
30002025-08-26T21:47:32.873ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 10 reason = Timeout
30012025-08-26T21:47:32.873ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
30022025-08-26T21:47:32.874ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
30032025-08-26T21:47:32.874ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
30042025-08-26T21:47:32.874ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
30052025-08-26T21:47:32.874ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
30062025-08-26T21:47:32.875ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
30072025-08-26T21:47:32.875ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30082025-08-26T21:47:32.876ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
30092025-08-26T21:47:32.877ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
30102025-08-26T21:47:32.879ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
30112025-08-26T21:47:32.879ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
30122025-08-26T21:47:32.879ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
30132025-08-26T21:47:32.882ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
30142025-08-26T21:47:32.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
30152025-08-26T21:47:32.884ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30162025-08-26T21:47:32.889ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
30172025-08-26T21:47:32.889ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
30182025-08-26T21:47:32.889ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
30192025-08-26T21:47:32.908ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
30202025-08-26T21:47:32.908ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 34.211801ms iteration = 10 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
30212025-08-26T21:47:32.910ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
30222025-08-26T21:47:32.910ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
30232025-08-26T21:47:32.911ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1213 zone_address = fd00:1122:3344:101::2
30242025-08-26T21:47:32.911ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
30252025-08-26T21:47:32.912ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "36c5d94c-0e48-4ebe-a438-20e37439c3a8", "content-length": "26", "date": "Tue, 26 Aug 2025 21:47:32 GMT"} })
30262025-08-26T21:47:32.912ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1216 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
30272025-08-26T21:47:32.912ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 50.136611ms iteration = 4 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
30282025-08-26T21:47:32.912ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
30292025-08-26T21:47:32.913ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
30302025-08-26T21:47:32.913ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
30312025-08-26T21:47:32.913ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 51.170443ms iteration = 4 status = {"errors":[],"finish_invoked_ok":[]}
30322025-08-26T21:47:32.917ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
30332025-08-26T21:47:32.917ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
30342025-08-26T21:47:32.917ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
30352025-08-26T21:47:32.918ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
30362025-08-26T21:47:32.918ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:550 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
30372025-08-26T21:47:32.918ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 57.422865ms iteration = 4 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
30382025-08-26T21:47:32.919ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
30392025-08-26T21:47:32.919ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
30402025-08-26T21:47:32.920ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30412025-08-26T21:47:32.923ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
30422025-08-26T21:47:32.923ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
30432025-08-26T21:47:32.923ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
30442025-08-26T21:47:32.923ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 60.942134ms iteration = 4 status = {"errors":[],"requests_created_ok":[]}
30452025-08-26T21:47:32.924ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1213 zone_address = fd00:1122:3344:101::2
30462025-08-26T21:47:32.924ZDEBGnexus (MgsClient): client request background_task = nat_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
30472025-08-26T21:47:32.924ZDEBGnexus (MgsClient): client response background_task = nat_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "aca5aeeb-b18c-4d44-bcc5-3aeaeab82eef", "content-length": "26", "date": "Tue, 26 Aug 2025 21:47:32 GMT"} })
30482025-08-26T21:47:32.924ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1216 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
30492025-08-26T21:47:32.926ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
30502025-08-26T21:47:32.926ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 64.995265ms iteration = 10 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
30512025-08-26T21:47:32.940ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 79.354199ms iteration = 5 status = {"physical_disks_added":0}
30522025-08-26T21:47:32.956ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1213 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 zone_address = fd00:1122:3344:101::2
30532025-08-26T21:47:32.956ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
30542025-08-26T21:47:32.956ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "03679ae0-e181-40f9-ba59-f110db64080f", "content-length": "26", "date": "Tue, 26 Aug 2025 21:47:32 GMT"} })
30552025-08-26T21:47:32.956ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1216 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
30562025-08-26T21:47:32.957ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
30572025-08-26T21:47:32.966ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
30582025-08-26T21:47:32.966ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
30592025-08-26T21:47:32.966ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 105.192541ms iteration = 4 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
30602025-08-26T21:47:32.967ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
30612025-08-26T21:47:32.968ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
30622025-08-26T21:47:32.969ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 106.799757ms iteration = 4 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
30632025-08-26T21:47:32.970ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:407
30642025-08-26T21:47:32.970ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 108.789172ms iteration = 4 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
30652025-08-26T21:47:33.014ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 179.466436ms iteration = 4 status = {}
30662025-08-26T21:47:33.015ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
30672025-08-26T21:47:33.015ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
30682025-08-26T21:47:33.016ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30692025-08-26T21:47:33.018ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
30702025-08-26T21:47:33.018ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
30712025-08-26T21:47:33.019ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
30722025-08-26T21:47:33.020ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
30732025-08-26T21:47:33.020ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
30742025-08-26T21:47:33.020ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
30752025-08-26T21:47:33.020ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
30762025-08-26T21:47:33.020ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30772025-08-26T21:47:33.022ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
30782025-08-26T21:47:33.022ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 187.879306ms iteration = 4 status = null
30792025-08-26T21:47:33.022ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
30802025-08-26T21:47:33.022ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
30812025-08-26T21:47:33.022ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
30822025-08-26T21:47:33.022ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
30832025-08-26T21:47:33.024ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30842025-08-26T21:47:33.027ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: dd0c655a-95c2-48fc-b4c3-9733ed4b0faa, lookup_type: ById(dd0c655a-95c2-48fc-b4c3-9733ed4b0faa) } result = Ok(())
30852025-08-26T21:47:33.027ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
30862025-08-26T21:47:33.028ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
30872025-08-26T21:47:33.029ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 168.076198ms iteration = 4 status = 0
30882025-08-26T21:47:33.054ZINFOnexus (BackgroundTasks): VPC route manager sled dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
30892025-08-26T21:47:33.054ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
30902025-08-26T21:47:33.054ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "943d0600-62fd-442f-aef1-8e1c7b86edc4", "content-length": "372", "date": "Tue, 26 Aug 2025 21:47:33 GMT"} })
30912025-08-26T21:47:33.054ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
30922025-08-26T21:47:33.055ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
30932025-08-26T21:47:33.069ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
30942025-08-26T21:47:33.069ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
30952025-08-26T21:47:33.070ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "4f104ebd-3d0d-4be5-a06b-0a078fdd65f3", "date": "Tue, 26 Aug 2025 21:47:33 GMT"} })
30962025-08-26T21:47:33.070ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
30972025-08-26T21:47:33.070ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
30982025-08-26T21:47:33.072ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
30992025-08-26T21:47:33.072ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
31002025-08-26T21:47:33.075ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
31012025-08-26T21:47:33.075ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
31022025-08-26T21:47:33.078ZINFOnexus (BackgroundTasks): VPC route manager sled dd0c655a-95c2-48fc-b4c3-9733ed4b0faa push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
31032025-08-26T21:47:33.078ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
31042025-08-26T21:47:33.078ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "f7c38f30-6e30-44be-9a9c-0c1294704bdb", "date": "Tue, 26 Aug 2025 21:47:33 GMT"} })
31052025-08-26T21:47:33.080ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 218.637629ms iteration = 4 status = {}
31062025-08-26T21:47:33.126ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
31072025-08-26T21:47:33.126ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
31082025-08-26T21:47:33.128ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: d24c5c55-3a21-4a91-aeb4-45362e822df6, rack_id: e37a46f2-f789-4ca4-9841-f8c6dd705964, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(88d84f37-a4c8-4abc-af1f-27987b0d39d9) } rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0
31092025-08-26T21:47:33.128ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
31102025-08-26T21:47:33.128ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
31112025-08-26T21:47:33.140ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1832 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
31122025-08-26T21:47:33.140ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1859 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 88d84f37-a4c8-4abc-af1f-27987b0d39d9
31132025-08-26T21:47:33.141ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
31142025-08-26T21:47:33.141ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
31152025-08-26T21:47:33.141ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
31162025-08-26T21:47:33.141ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
31172025-08-26T21:47:33.141ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
31182025-08-26T21:47:33.141ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2006 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
31192025-08-26T21:47:33.141ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:408 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
31202025-08-26T21:47:33.141ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2042 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
31212025-08-26T21:47:33.141ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
31222025-08-26T21:47:33.141ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
31232025-08-26T21:47:33.141ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
31242025-08-26T21:47:33.142ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
31252025-08-26T21:47:33.142ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "965e1f05-271e-4d87-8d1a-ce228799de76", "content-length": "691", "date": "Tue, 26 Aug 2025 21:47:33 GMT"} })
31262025-08-26T21:47:33.142ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
31272025-08-26T21:47:33.142ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
31282025-08-26T21:47:33.142ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
31292025-08-26T21:47:33.143ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0
31302025-08-26T21:47:33.143ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
31312025-08-26T21:47:33.144ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 282.799325ms iteration = 4 status = {}
31322025-08-26T21:47:33.265ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "2461d405-ac7c-45eb-9733-0752d9ccfae1", "date": "Tue, 26 Aug 2025 21:47:33 GMT"} })
31332025-08-26T21:47:33.265ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch = Switch0
31342025-08-26T21:47:33.265ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
31352025-08-26T21:47:33.265ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "08ca58e1-aa1d-4c32-8218-02789c5addd8", "content-length": "682", "date": "Tue, 26 Aug 2025 21:47:33 GMT"} })
31362025-08-26T21:47:33.265ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
31372025-08-26T21:47:33.265ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
31382025-08-26T21:47:33.269ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
31392025-08-26T21:47:33.269ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
31402025-08-26T21:47:33.273ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
31412025-08-26T21:47:33.274ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
31422025-08-26T21:47:33.275ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
31432025-08-26T21:47:33.276ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
31442025-08-26T21:47:33.282ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 420.621551ms iteration = 4 status = {}
31452025-08-26T21:47:35.353ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 289 local_addr = [fd00:1122:3344:101::c]:42016 method = GET remote_addr = [fd00:1122:3344:101::d]:37550 req_id = 45f39f03-738b-42a1-ba76-c20f400cb8c9 response_code = 200 uri = /5d0959e5-47a6-4bf3-a52b-57950b9d900d
31462025-08-26T21:47:37.848ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 20 reason = Timeout
31472025-08-26T21:47:37.848ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
31482025-08-26T21:47:37.848ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
31492025-08-26T21:47:37.850ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31502025-08-26T21:47:37.853ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
31512025-08-26T21:47:37.853ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
31522025-08-26T21:47:37.853ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
31532025-08-26T21:47:37.855ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
31542025-08-26T21:47:37.855ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 6.906526ms iteration = 20 status = {"chicken_switches_updated":false}
31552025-08-26T21:47:42.848ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 21 reason = Timeout
31562025-08-26T21:47:42.848ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
31572025-08-26T21:47:42.848ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
31582025-08-26T21:47:42.850ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31592025-08-26T21:47:42.853ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
31602025-08-26T21:47:42.853ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
31612025-08-26T21:47:42.853ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
31622025-08-26T21:47:42.855ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
31632025-08-26T21:47:42.855ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 6.932426ms iteration = 21 status = {"chicken_switches_updated":false}
31642025-08-26T21:47:42.861ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 11 reason = Timeout
31652025-08-26T21:47:42.861ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
31662025-08-26T21:47:42.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
31672025-08-26T21:47:42.873ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 11 reason = Timeout
31682025-08-26T21:47:42.873ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
31692025-08-26T21:47:42.873ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
31702025-08-26T21:47:42.875ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31712025-08-26T21:47:42.877ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
31722025-08-26T21:47:42.877ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
31732025-08-26T21:47:42.878ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
31742025-08-26T21:47:42.880ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
31752025-08-26T21:47:42.880ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
31762025-08-26T21:47:42.881ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31772025-08-26T21:47:42.884ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
31782025-08-26T21:47:42.884ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
31792025-08-26T21:47:42.884ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
31802025-08-26T21:47:42.898ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
31812025-08-26T21:47:42.898ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
31822025-08-26T21:47:42.942ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 81.213041ms iteration = 11 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
31832025-08-26T21:47:42.997ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
31842025-08-26T21:47:42.997ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 124.291582ms iteration = 11 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
31852025-08-26T21:47:45.353ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 290 local_addr = [fd00:1122:3344:101::c]:42016 method = GET remote_addr = [fd00:1122:3344:101::d]:37550 req_id = e08e5b0b-db4b-48a3-bfca-3dc73a402d00 response_code = 200 uri = /5d0959e5-47a6-4bf3-a52b-57950b9d900d
31862025-08-26T21:47:47.847ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 22 reason = Timeout
31872025-08-26T21:47:47.848ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
31882025-08-26T21:47:47.848ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
31892025-08-26T21:47:47.849ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31902025-08-26T21:47:47.852ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
31912025-08-26T21:47:47.852ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
31922025-08-26T21:47:47.853ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
31932025-08-26T21:47:47.855ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
31942025-08-26T21:47:47.855ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.123081ms iteration = 22 status = {"chicken_switches_updated":false}
31952025-08-26T21:47:52.848ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 23 reason = Timeout
31962025-08-26T21:47:52.848ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
31972025-08-26T21:47:52.848ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
31982025-08-26T21:47:52.850ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31992025-08-26T21:47:52.853ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
32002025-08-26T21:47:52.853ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
32012025-08-26T21:47:52.853ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
32022025-08-26T21:47:52.855ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
32032025-08-26T21:47:52.855ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.453558ms iteration = 23 status = {"chicken_switches_updated":false}
32042025-08-26T21:47:52.861ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 12 reason = Timeout
32052025-08-26T21:47:52.861ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
32062025-08-26T21:47:52.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
32072025-08-26T21:47:52.864ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
32082025-08-26T21:47:52.864ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
32092025-08-26T21:47:52.866ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.508262ms iteration = 12 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
32102025-08-26T21:47:52.873ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 12 reason = Timeout
32112025-08-26T21:47:52.873ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
32122025-08-26T21:47:52.873ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
32132025-08-26T21:47:52.875ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32142025-08-26T21:47:52.878ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
32152025-08-26T21:47:52.878ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
32162025-08-26T21:47:52.878ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
32172025-08-26T21:47:52.879ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
32182025-08-26T21:47:52.879ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
32192025-08-26T21:47:52.881ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32202025-08-26T21:47:52.884ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
32212025-08-26T21:47:52.884ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
32222025-08-26T21:47:52.884ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
32232025-08-26T21:47:52.897ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
32242025-08-26T21:47:52.897ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 23.907449ms iteration = 12 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
32252025-08-26T21:47:55.353ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 270 local_addr = [fd00:1122:3344:101::c]:42016 method = GET remote_addr = [fd00:1122:3344:101::d]:37550 req_id = 7c364077-bd25-43fd-b1f6-efbe47668e5e response_code = 200 uri = /5d0959e5-47a6-4bf3-a52b-57950b9d900d
32262025-08-26T21:47:57.848ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 24 reason = Timeout
32272025-08-26T21:47:57.849ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
32282025-08-26T21:47:57.849ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
32292025-08-26T21:47:57.851ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32302025-08-26T21:47:57.853ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
32312025-08-26T21:47:57.854ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
32322025-08-26T21:47:57.854ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
32332025-08-26T21:47:57.856ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
32342025-08-26T21:47:57.856ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.206122ms iteration = 24 status = {"chicken_switches_updated":false}
32352025-08-26T21:48:02.834ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 5 reason = Timeout
32362025-08-26T21:48:02.834ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 3 reason = Timeout
32372025-08-26T21:48:02.834ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
32382025-08-26T21:48:02.834ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
32392025-08-26T21:48:02.834ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 5 reason = Timeout
32402025-08-26T21:48:02.834ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 3 reason = Timeout
32412025-08-26T21:48:02.834ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
32422025-08-26T21:48:02.834ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
32432025-08-26T21:48:02.834ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 3 reason = Timeout
32442025-08-26T21:48:02.834ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 5 reason = Timeout
32452025-08-26T21:48:02.834ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 3 reason = Timeout
32462025-08-26T21:48:02.834ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 5 reason = Timeout
32472025-08-26T21:48:02.834ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
32482025-08-26T21:48:02.834ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
32492025-08-26T21:48:02.834ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 5 reason = Timeout
32502025-08-26T21:48:02.834ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
32512025-08-26T21:48:02.834ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 3 reason = Timeout
32522025-08-26T21:48:02.834ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 3 reason = Timeout
32532025-08-26T21:48:02.834ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-08-26 21:38:02.834415584 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
32542025-08-26T21:48:02.834ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
32552025-08-26T21:48:02.834ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
32562025-08-26T21:48:02.834ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
32572025-08-26T21:48:02.834ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
32582025-08-26T21:48:02.834ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
32592025-08-26T21:48:02.834ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("60abf731-31e1-4111-a3e2-65da89eaed87.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("84c849eb-0949-41d9-ad02-9a5509144564.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("84c849eb-0949-41d9-ad02-9a5509144564.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::8)) }], valid_until: Instant { tv_sec: 986, tv_nsec: 692304561 } })
32602025-08-26T21:48:02.834ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
32612025-08-26T21:48:02.834ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
32622025-08-26T21:48:02.835ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("852fd1a3-de9a-47e8-a667-5e8dab880ad3.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("c68f1501-294c-4874-92ca-8fc0546a5870.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("d741a72f-8745-4723-9e80-e1e72987266b.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("d741a72f-8745-4723-9e80-e1e72987266b.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:3::1)) }], valid_until: Instant { tv_sec: 986, tv_nsec: 692432174 } })
32632025-08-26T21:48:02.835ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
32642025-08-26T21:48:02.835ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 945.361\u{b5}s iteration = 3 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
32652025-08-26T21:48:02.835ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } dns_group = internal
32662025-08-26T21:48:02.835ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.068544ms iteration = 3 status = {"addresses":["[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:3::1]:5353"]}
32672025-08-26T21:48:02.835ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
32682025-08-26T21:48:02.835ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
32692025-08-26T21:48:02.835ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
32702025-08-26T21:48:02.836ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
32712025-08-26T21:48:02.836ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32722025-08-26T21:48:02.836ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 4 reason = Timeout
32732025-08-26T21:48:02.837ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:91 reason = blueprint disabled target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8
32742025-08-26T21:48:02.837ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 14.71\u{b5}s iteration = 4 status = {"enabled":false,"target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8"}
32752025-08-26T21:48:02.837ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-08-26 21:38:02.834415584 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
32762025-08-26T21:48:02.837ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 2.842044ms iteration = 3 status = {"expiration":"2025-08-26T21:38:02.834415584Z","pruned":[]}
32772025-08-26T21:48:02.837ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 986, tv_nsec: 694705045 } })
32782025-08-26T21:48:02.837ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1205
32792025-08-26T21:48:02.837ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32802025-08-26T21:48:02.837ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32812025-08-26T21:48:02.838ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.821036ms iteration = 5 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
32822025-08-26T21:48:02.840ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
32832025-08-26T21:48:02.840ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
32842025-08-26T21:48:02.840ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
32852025-08-26T21:48:02.841ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
32862025-08-26T21:48:02.842ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
32872025-08-26T21:48:02.843ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
32882025-08-26T21:48:02.843ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
32892025-08-26T21:48:02.844ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
32902025-08-26T21:48:02.844ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
32912025-08-26T21:48:02.844ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
32922025-08-26T21:48:02.844ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
32932025-08-26T21:48:02.844ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32942025-08-26T21:48:02.844ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
32952025-08-26T21:48:02.844ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 986, tv_nsec: 702338347 } })
32962025-08-26T21:48:02.845ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1205
32972025-08-26T21:48:02.846ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32982025-08-26T21:48:02.847ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
32992025-08-26T21:48:02.847ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
33002025-08-26T21:48:02.848ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
33012025-08-26T21:48:02.848ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 25 reason = Timeout
33022025-08-26T21:48:02.848ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
33032025-08-26T21:48:02.848ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
33042025-08-26T21:48:02.849ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
33052025-08-26T21:48:02.849ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33062025-08-26T21:48:02.849ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
33072025-08-26T21:48:02.850ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
33082025-08-26T21:48:02.850ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
33092025-08-26T21:48:02.850ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
33102025-08-26T21:48:02.851ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33112025-08-26T21:48:02.852ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
33122025-08-26T21:48:02.852ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = dfb404ab-acb8-43de-afe8-b429af0c161d dns_zone_name = oxide.test found_so_far = 0 version = 2
33132025-08-26T21:48:02.852ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
33142025-08-26T21:48:02.852ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
33152025-08-26T21:48:02.852ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
33162025-08-26T21:48:02.852ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
33172025-08-26T21:48:02.853ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
33182025-08-26T21:48:02.853ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
33192025-08-26T21:48:02.853ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
33202025-08-26T21:48:02.854ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
33212025-08-26T21:48:02.854ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
33222025-08-26T21:48:02.854ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33232025-08-26T21:48:02.854ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 5.894733ms iteration = 25 status = {"chicken_switches_updated":false}
33242025-08-26T21:48:02.857ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
33252025-08-26T21:48:02.857ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
33262025-08-26T21:48:02.857ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
33272025-08-26T21:48:02.858ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = dfb404ab-acb8-43de-afe8-b429af0c161d dns_zone_name = oxide.test found_so_far = 4 version = 2
33282025-08-26T21:48:02.858ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
33292025-08-26T21:48:02.858ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2025-08-26 21:46:05.296205 UTC dns_group = external generation = 2
33302025-08-26T21:48:02.858ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 24.530153ms iteration = 3 status = {"generation":2}
33312025-08-26T21:48:02.861ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 5 reason = Timeout
33322025-08-26T21:48:02.861ZDEBGnexus (BackgroundTasks): blueprint planning disabled, doing nothing background_task = blueprint_planner
33332025-08-26T21:48:02.861ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 5 reason = Timeout
33342025-08-26T21:48:02.861ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 14.64\u{b5}s iteration = 5 status = "Disabled"
33352025-08-26T21:48:02.861ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
33362025-08-26T21:48:02.861ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
33372025-08-26T21:48:02.861ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 5 reason = Timeout
33382025-08-26T21:48:02.861ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 5 reason = Timeout
33392025-08-26T21:48:02.861ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
33402025-08-26T21:48:02.861ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
33412025-08-26T21:48:02.861ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
33422025-08-26T21:48:02.861ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 5 reason = Timeout
33432025-08-26T21:48:02.861ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
33442025-08-26T21:48:02.861ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 5 reason = Timeout
33452025-08-26T21:48:02.861ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
33462025-08-26T21:48:02.861ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 13 reason = Timeout
33472025-08-26T21:48:02.861ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
33482025-08-26T21:48:02.861ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 3 reason = Timeout
33492025-08-26T21:48:02.861ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
33502025-08-26T21:48:02.861ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 3 reason = Timeout
33512025-08-26T21:48:02.861ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
33522025-08-26T21:48:02.861ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 5 reason = Timeout
33532025-08-26T21:48:02.861ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 3 reason = Timeout
33542025-08-26T21:48:02.861ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:53
33552025-08-26T21:48:02.861ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
33562025-08-26T21:48:02.861ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
33572025-08-26T21:48:02.861ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 5 reason = Timeout
33582025-08-26T21:48:02.861ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
33592025-08-26T21:48:02.861ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 6 reason = Timeout
33602025-08-26T21:48:02.861ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
33612025-08-26T21:48:02.862ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 5 reason = Timeout
33622025-08-26T21:48:02.862ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
33632025-08-26T21:48:02.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
33642025-08-26T21:48:02.862ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 5 reason = Timeout
33652025-08-26T21:48:02.862ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 3 reason = Timeout
33662025-08-26T21:48:02.862ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
33672025-08-26T21:48:02.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
33682025-08-26T21:48:02.862ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
33692025-08-26T21:48:02.862ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 5 reason = Timeout
33702025-08-26T21:48:02.862ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
33712025-08-26T21:48:02.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
33722025-08-26T21:48:02.862ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 5 reason = Timeout
33732025-08-26T21:48:02.862ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
33742025-08-26T21:48:02.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
33752025-08-26T21:48:02.862ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 5 reason = Timeout
33762025-08-26T21:48:02.862ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
33772025-08-26T21:48:02.862ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 5 reason = Timeout
33782025-08-26T21:48:02.862ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
33792025-08-26T21:48:02.862ZDEBGnexus (BackgroundTasks): activating background_task = alert_dispatcher iteration = 3 reason = Timeout
33802025-08-26T21:48:02.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
33812025-08-26T21:48:02.862ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
33822025-08-26T21:48:02.862ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 5 reason = Timeout
33832025-08-26T21:48:02.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
33842025-08-26T21:48:02.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
33852025-08-26T21:48:02.862ZDEBGnexus (BackgroundTasks): activating background_task = webhook_deliverator iteration = 3 reason = Timeout
33862025-08-26T21:48:02.862ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 17.35\u{b5}s iteration = 5 status = {"disabled":true,"errors":[],"sps":[]}
33872025-08-26T21:48:02.862ZDEBGnexus (BackgroundTasks): roles background_task = webhook_deliverator roles = RoleSet { roles: {} }
33882025-08-26T21:48:02.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
33892025-08-26T21:48:02.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
33902025-08-26T21:48:02.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
33912025-08-26T21:48:02.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
33922025-08-26T21:48:02.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
33932025-08-26T21:48:02.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
33942025-08-26T21:48:02.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
33952025-08-26T21:48:02.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
33962025-08-26T21:48:02.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
33972025-08-26T21:48:02.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
33982025-08-26T21:48:02.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
33992025-08-26T21:48:02.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = webhook_deliverator resource = Database result = Ok(())
34002025-08-26T21:48:02.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
34012025-08-26T21:48:02.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = alert_dispatcher resource = Database result = Ok(())
34022025-08-26T21:48:02.865ZINFOnexus (BackgroundTasks): all webhook delivery tasks completed background_task = webhook_deliverator file = nexus/src/app/background/tasks/webhook_deliverator.rs:204 num_receivers = 0
34032025-08-26T21:48:02.865ZDEBGnexus (BackgroundTasks): activation complete background_task = webhook_deliverator elapsed = 2.926266ms iteration = 3 status = {"by_rx":{},"error":null}
34042025-08-26T21:48:02.865ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 3.412347ms iteration = 5 status = {"errors":[],"finish_invoked_ok":[]}
34052025-08-26T21:48:02.865ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34062025-08-26T21:48:02.865ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34072025-08-26T21:48:02.866ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 4.535432ms iteration = 5 status = {"errors":[],"garbage_collect_requested":[]}
34082025-08-26T21:48:02.866ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 4.775757ms iteration = 3 status = {"errors":[],"found_port_ok":[]}
34092025-08-26T21:48:02.867ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
34102025-08-26T21:48:02.867ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
34112025-08-26T21:48:02.867ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34122025-08-26T21:48:02.867ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
34132025-08-26T21:48:02.867ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
34142025-08-26T21:48:02.867ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
34152025-08-26T21:48:02.867ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34162025-08-26T21:48:02.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
34172025-08-26T21:48:02.868ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34182025-08-26T21:48:02.868ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
34192025-08-26T21:48:02.868ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:170 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
34202025-08-26T21:48:02.868ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 7.052229ms iteration = 3 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
34212025-08-26T21:48:02.868ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
34222025-08-26T21:48:02.868ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
34232025-08-26T21:48:02.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
34242025-08-26T21:48:02.869ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
34252025-08-26T21:48:02.869ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
34262025-08-26T21:48:02.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
34272025-08-26T21:48:02.871ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
34282025-08-26T21:48:02.871ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
34292025-08-26T21:48:02.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
34302025-08-26T21:48:02.872ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
34312025-08-26T21:48:02.872ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
34322025-08-26T21:48:02.872ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
34332025-08-26T21:48:02.872ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
34342025-08-26T21:48:02.872ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
34352025-08-26T21:48:02.873ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 13 reason = Timeout
34362025-08-26T21:48:02.873ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
34372025-08-26T21:48:02.873ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
34382025-08-26T21:48:02.873ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 986, tv_nsec: 730906621 } })
34392025-08-26T21:48:02.873ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
34402025-08-26T21:48:02.874ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1205 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
34412025-08-26T21:48:02.874ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34422025-08-26T21:48:02.876ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
34432025-08-26T21:48:02.876ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34442025-08-26T21:48:02.876ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
34452025-08-26T21:48:02.877ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
34462025-08-26T21:48:02.877ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
34472025-08-26T21:48:02.877ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
34482025-08-26T21:48:02.877ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
34492025-08-26T21:48:02.877ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
34502025-08-26T21:48:02.877ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
34512025-08-26T21:48:02.877ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
34522025-08-26T21:48:02.877ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
34532025-08-26T21:48:02.877ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
34542025-08-26T21:48:02.878ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:550 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
34552025-08-26T21:48:02.878ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 16.41651ms iteration = 5 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
34562025-08-26T21:48:02.878ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34572025-08-26T21:48:02.879ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34582025-08-26T21:48:02.880ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
34592025-08-26T21:48:02.880ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
34602025-08-26T21:48:02.880ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
34612025-08-26T21:48:02.881ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
34622025-08-26T21:48:02.881ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
34632025-08-26T21:48:02.881ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
34642025-08-26T21:48:02.881ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
34652025-08-26T21:48:02.881ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
34662025-08-26T21:48:02.881ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 19.733795ms iteration = 5 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
34672025-08-26T21:48:02.882ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
34682025-08-26T21:48:02.882ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
34692025-08-26T21:48:02.882ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
34702025-08-26T21:48:02.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
34712025-08-26T21:48:02.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
34722025-08-26T21:48:02.884ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:407
34732025-08-26T21:48:02.884ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 22.902947ms iteration = 5 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
34742025-08-26T21:48:02.884ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34752025-08-26T21:48:02.889ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
34762025-08-26T21:48:02.889ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
34772025-08-26T21:48:02.889ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
34782025-08-26T21:48:02.892ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
34792025-08-26T21:48:02.892ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = alert_dispatcher resource = Database result = Ok(())
34802025-08-26T21:48:02.895ZDEBGnexus (BackgroundTasks): activation complete background_task = alert_dispatcher elapsed = 33.471945ms iteration = 3 status = {"dispatched":[],"errors":[],"glob_version":"183.0.0","globs_reprocessed":{},"no_receivers":[]}
34812025-08-26T21:48:02.899ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 37.490205ms iteration = 6 status = {"physical_disks_added":0}
34822025-08-26T21:48:02.900ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
34832025-08-26T21:48:02.900ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
34842025-08-26T21:48:02.900ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
34852025-08-26T21:48:02.900ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
34862025-08-26T21:48:02.900ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
34872025-08-26T21:48:02.902ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34882025-08-26T21:48:02.905ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
34892025-08-26T21:48:02.905ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
34902025-08-26T21:48:02.905ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
34912025-08-26T21:48:02.908ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
34922025-08-26T21:48:02.908ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 8e720f02-dde8-4423-a6e3-91698373805a dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
34932025-08-26T21:48:02.908ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
34942025-08-26T21:48:02.908ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
34952025-08-26T21:48:02.909ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
34962025-08-26T21:48:02.909ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
34972025-08-26T21:48:02.910ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34982025-08-26T21:48:02.911ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
34992025-08-26T21:48:02.911ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 49.26968ms iteration = 5 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
35002025-08-26T21:48:02.911ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
35012025-08-26T21:48:02.912ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35022025-08-26T21:48:02.916ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
35032025-08-26T21:48:02.916ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
35042025-08-26T21:48:02.916ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
35052025-08-26T21:48:02.916ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
35062025-08-26T21:48:02.916ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
35072025-08-26T21:48:02.917ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
35082025-08-26T21:48:02.917ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
35092025-08-26T21:48:02.917ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
35102025-08-26T21:48:02.917ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
35112025-08-26T21:48:02.917ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
35122025-08-26T21:48:02.918ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 8e720f02-dde8-4423-a6e3-91698373805a dns_zone_name = control-plane.oxide.internal found_so_far = 59 version = 1
35132025-08-26T21:48:02.918ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
35142025-08-26T21:48:02.918ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2025-08-26 21:46:04.065217 UTC dns_group = internal generation = 1
35152025-08-26T21:48:02.918ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 84.537046ms iteration = 3 status = {"generation":1}
35162025-08-26T21:48:02.922ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 60.881753ms iteration = 5 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
35172025-08-26T21:48:02.923ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 61.532167ms iteration = 13 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
35182025-08-26T21:48:02.928ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
35192025-08-26T21:48:02.928ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-08-26T21:45:59.866217Z, time_modified: 2025-08-26T21:45:59.866217Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 9396aa55-de6f-4d6f-96db-d28e1c76fa9c, db_silo: Silo { identity: SiloIdentity { id: 9396aa55-de6f-4d6f-96db-d28e1c76fa9c, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-08-26T21:46:05.245521Z, time_modified: 2025-08-26T21:46:05.245521Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "e8529d720e4b0de099753dcdb373abb870af04baacd7c7dd850f5254858081af" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 9396aa55-de6f-4d6f-96db-d28e1c76fa9c, db_silo: Silo { identity: SiloIdentity { id: 9396aa55-de6f-4d6f-96db-d28e1c76fa9c, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-08-26T21:46:05.245521Z, time_modified: 2025-08-26T21:46:05.245521Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "e8529d720e4b0de099753dcdb373abb870af04baacd7c7dd850f5254858081af" }] }) }
35202025-08-26T21:48:02.928ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 94.03995ms iteration = 3 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"9396aa55-de6f-4d6f-96db-d28e1c76fa9c","tls_certs":["e8529d720e4b0de099753dcdb373abb870af04baacd7c7dd850f5254858081af"]}},"default_endpoint":{"silo_id":"9396aa55-de6f-4d6f-96db-d28e1c76fa9c","tls_certs":["e8529d720e4b0de099753dcdb373abb870af04baacd7c7dd850f5254858081af"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
35212025-08-26T21:48:02.928ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
35222025-08-26T21:48:02.929ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 67.502821ms iteration = 5 status = {"errors":[],"requests_created_ok":[]}
35232025-08-26T21:48:02.933ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35242025-08-26T21:48:02.935ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
35252025-08-26T21:48:02.935ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
35262025-08-26T21:48:02.935ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
35272025-08-26T21:48:02.961ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 99.221907ms iteration = 5 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
35282025-08-26T21:48:02.968ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
35292025-08-26T21:48:02.971ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
35302025-08-26T21:48:02.971ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
35312025-08-26T21:48:02.971ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
35322025-08-26T21:48:02.974ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
35332025-08-26T21:48:02.974ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
35342025-08-26T21:48:02.975ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35352025-08-26T21:48:02.975ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
35362025-08-26T21:48:02.975ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:138 found = 0 not_ready_to_be_deleted = 0
35372025-08-26T21:48:02.975ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 114.053751ms iteration = 3 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
35382025-08-26T21:48:02.978ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: dd0c655a-95c2-48fc-b4c3-9733ed4b0faa, lookup_type: ById(dd0c655a-95c2-48fc-b4c3-9733ed4b0faa) } result = Ok(())
35392025-08-26T21:48:02.978ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
35402025-08-26T21:48:02.978ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
35412025-08-26T21:48:02.980ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 119.197247ms iteration = 5 status = 0
35422025-08-26T21:48:02.991ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1213 zone_address = fd00:1122:3344:101::2
35432025-08-26T21:48:02.991ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
35442025-08-26T21:48:02.992ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "9423e110-eda4-4c2f-a385-8fc18988bded", "content-length": "26", "date": "Tue, 26 Aug 2025 21:48:02 GMT"} })
35452025-08-26T21:48:02.992ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1216 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
35462025-08-26T21:48:02.993ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
35472025-08-26T21:48:03.009ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
35482025-08-26T21:48:03.009ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
35492025-08-26T21:48:03.010ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
35502025-08-26T21:48:03.010ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 137.350306ms iteration = 13 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
35512025-08-26T21:48:03.013ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
35522025-08-26T21:48:03.014ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
35532025-08-26T21:48:03.018ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
35542025-08-26T21:48:03.019ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
35552025-08-26T21:48:03.020ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35562025-08-26T21:48:03.023ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
35572025-08-26T21:48:03.023ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
35582025-08-26T21:48:03.023ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
35592025-08-26T21:48:03.025ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35602025-08-26T21:48:03.027ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
35612025-08-26T21:48:03.027ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
35622025-08-26T21:48:03.028ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
35632025-08-26T21:48:03.038ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1213 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 zone_address = fd00:1122:3344:101::2
35642025-08-26T21:48:03.038ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
35652025-08-26T21:48:03.039ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "d9bd9351-30b3-4534-8874-ddb9deb68426", "content-length": "26", "date": "Tue, 26 Aug 2025 21:48:02 GMT"} })
35662025-08-26T21:48:03.039ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1216 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
35672025-08-26T21:48:03.040ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
35682025-08-26T21:48:03.069ZINFOnexus (BackgroundTasks): instance reincarnation completed successfully background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:113 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0
35692025-08-26T21:48:03.069ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 207.883886ms iteration = 3 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
35702025-08-26T21:48:03.098ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1213 zone_address = fd00:1122:3344:101::2
35712025-08-26T21:48:03.098ZDEBGnexus (MgsClient): client request background_task = nat_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
35722025-08-26T21:48:03.099ZDEBGnexus (MgsClient): client response background_task = nat_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "d0d5b0d0-f2c1-44e2-8bd7-6fc0851dca1c", "content-length": "26", "date": "Tue, 26 Aug 2025 21:48:02 GMT"} })
35732025-08-26T21:48:03.099ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1216 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
35742025-08-26T21:48:03.100ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
35752025-08-26T21:48:03.156ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
35762025-08-26T21:48:03.157ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "49ff09fa-096a-4f61-8963-473c2101d677", "date": "Tue, 26 Aug 2025 21:48:03 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
35772025-08-26T21:48:03.157ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
35782025-08-26T21:48:03.158ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "d0c9cdbe-956a-4ca3-bb15-37681cda55eb", "date": "Tue, 26 Aug 2025 21:48:03 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
35792025-08-26T21:48:03.158ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
35802025-08-26T21:48:03.160ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
35812025-08-26T21:48:03.160ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 325.962628ms iteration = 5 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
35822025-08-26T21:48:03.173ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
35832025-08-26T21:48:03.175ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 341.017617ms iteration = 5 status = {}
35842025-08-26T21:48:03.190ZINFOnexus (BackgroundTasks): VPC route manager sled dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
35852025-08-26T21:48:03.190ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
35862025-08-26T21:48:03.190ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "8cbeaa74-bbb1-4145-8fc9-673e461ac460", "content-length": "372", "date": "Tue, 26 Aug 2025 21:48:03 GMT"} })
35872025-08-26T21:48:03.190ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
35882025-08-26T21:48:03.191ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
35892025-08-26T21:48:03.203ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
35902025-08-26T21:48:03.203ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "1201470a-6b88-4a04-acc8-0420c162bed2", "content-length": "691", "date": "Tue, 26 Aug 2025 21:48:03 GMT"} })
35912025-08-26T21:48:03.203ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
35922025-08-26T21:48:03.203ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
35932025-08-26T21:48:03.203ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
35942025-08-26T21:48:03.205ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 343.412131ms iteration = 5 status = {}
35952025-08-26T21:48:03.207ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
35962025-08-26T21:48:03.207ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
35972025-08-26T21:48:03.207ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "1cd96d81-7507-4ab5-a6ed-8f4104e2e507", "date": "Tue, 26 Aug 2025 21:48:03 GMT"} })
35982025-08-26T21:48:03.207ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
35992025-08-26T21:48:03.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
36002025-08-26T21:48:03.210ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
36012025-08-26T21:48:03.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
36022025-08-26T21:48:03.213ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
36032025-08-26T21:48:03.213ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
36042025-08-26T21:48:03.217ZINFOnexus (BackgroundTasks): VPC route manager sled dd0c655a-95c2-48fc-b4c3-9733ed4b0faa push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
36052025-08-26T21:48:03.217ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
36062025-08-26T21:48:03.217ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "e822e897-fd9c-4453-88e3-2fae0a03e478", "date": "Tue, 26 Aug 2025 21:48:03 GMT"} })
36072025-08-26T21:48:03.219ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 357.767065ms iteration = 5 status = {}
36082025-08-26T21:48:03.250ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
36092025-08-26T21:48:03.250ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
36102025-08-26T21:48:03.253ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 418.727829ms iteration = 5 status = null
36112025-08-26T21:48:03.258ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
36122025-08-26T21:48:03.258ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "fed8be3f-3a0d-4e97-98dd-126b8684b476", "date": "Tue, 26 Aug 2025 21:48:03 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
36132025-08-26T21:48:03.258ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
36142025-08-26T21:48:03.259ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "ec1ef40e-39f2-4968-9ac6-6b297c8b0819", "date": "Tue, 26 Aug 2025 21:48:03 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
36152025-08-26T21:48:03.259ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
36162025-08-26T21:48:03.261ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "08f1d17a-6a80-4e87-8e42-07b1d728a7df", "date": "Tue, 26 Aug 2025 21:48:03 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
36172025-08-26T21:48:03.261ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
36182025-08-26T21:48:03.263ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
36192025-08-26T21:48:03.263ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 428.883288ms iteration = 5 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
36202025-08-26T21:48:03.297ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
36212025-08-26T21:48:03.297ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
36222025-08-26T21:48:03.299ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: d24c5c55-3a21-4a91-aeb4-45362e822df6, rack_id: e37a46f2-f789-4ca4-9841-f8c6dd705964, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(88d84f37-a4c8-4abc-af1f-27987b0d39d9) } rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0
36232025-08-26T21:48:03.299ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
36242025-08-26T21:48:03.299ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
36252025-08-26T21:48:03.308ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1832 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
36262025-08-26T21:48:03.308ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1859 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 88d84f37-a4c8-4abc-af1f-27987b0d39d9
36272025-08-26T21:48:03.308ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
36282025-08-26T21:48:03.308ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
36292025-08-26T21:48:03.308ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
36302025-08-26T21:48:03.308ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
36312025-08-26T21:48:03.308ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
36322025-08-26T21:48:03.308ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2006 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
36332025-08-26T21:48:03.308ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:408 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
36342025-08-26T21:48:03.308ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2042 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
36352025-08-26T21:48:03.309ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
36362025-08-26T21:48:03.309ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
36372025-08-26T21:48:03.309ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
36382025-08-26T21:48:03.311ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0
36392025-08-26T21:48:03.311ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
36402025-08-26T21:48:03.413ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "20ccd003-4663-421d-9fbd-fc5dab9c4add", "date": "Tue, 26 Aug 2025 21:48:03 GMT"} })
36412025-08-26T21:48:03.413ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch = Switch0
36422025-08-26T21:48:03.413ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
36432025-08-26T21:48:03.413ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "668f141e-8715-42fb-b28f-0632a3db77f6", "content-length": "682", "date": "Tue, 26 Aug 2025 21:48:03 GMT"} })
36442025-08-26T21:48:03.413ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
36452025-08-26T21:48:03.413ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
36462025-08-26T21:48:03.416ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
36472025-08-26T21:48:03.417ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
36482025-08-26T21:48:03.420ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
36492025-08-26T21:48:03.420ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
36502025-08-26T21:48:03.422ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
36512025-08-26T21:48:03.422ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
36522025-08-26T21:48:03.429ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 567.770839ms iteration = 5 status = {}
36532025-08-26T21:48:05.353ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 316 local_addr = [fd00:1122:3344:101::c]:42016 method = GET remote_addr = [fd00:1122:3344:101::d]:37550 req_id = 4f89eac0-200d-4917-a881-5ea70fce0c96 response_code = 200 uri = /5d0959e5-47a6-4bf3-a52b-57950b9d900d
36542025-08-26T21:48:07.848ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 26 reason = Timeout
36552025-08-26T21:48:07.848ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
36562025-08-26T21:48:07.849ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
36572025-08-26T21:48:07.850ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36582025-08-26T21:48:07.853ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
36592025-08-26T21:48:07.853ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
36602025-08-26T21:48:07.854ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
36612025-08-26T21:48:07.855ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
36622025-08-26T21:48:07.855ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 6.870685ms iteration = 26 status = {"chicken_switches_updated":false}
36632025-08-26T21:48:12.848ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 27 reason = Timeout
36642025-08-26T21:48:12.848ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
36652025-08-26T21:48:12.849ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
36662025-08-26T21:48:12.851ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36672025-08-26T21:48:12.854ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
36682025-08-26T21:48:12.854ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
36692025-08-26T21:48:12.854ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
36702025-08-26T21:48:12.856ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
36712025-08-26T21:48:12.856ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.56803ms iteration = 27 status = {"chicken_switches_updated":false}
36722025-08-26T21:48:12.861ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 14 reason = Timeout
36732025-08-26T21:48:12.861ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
36742025-08-26T21:48:12.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
36752025-08-26T21:48:12.863ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
36762025-08-26T21:48:12.863ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
36772025-08-26T21:48:12.865ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.215895ms iteration = 14 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
36782025-08-26T21:48:12.873ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 14 reason = Timeout
36792025-08-26T21:48:12.873ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
36802025-08-26T21:48:12.873ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
36812025-08-26T21:48:12.875ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36822025-08-26T21:48:12.878ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
36832025-08-26T21:48:12.878ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
36842025-08-26T21:48:12.878ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
36852025-08-26T21:48:12.879ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
36862025-08-26T21:48:12.879ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
36872025-08-26T21:48:12.881ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36882025-08-26T21:48:12.884ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
36892025-08-26T21:48:12.884ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
36902025-08-26T21:48:12.884ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
36912025-08-26T21:48:12.985ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
36922025-08-26T21:48:12.985ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 111.957774ms iteration = 14 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
36932025-08-26T21:48:15.353ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 297 local_addr = [fd00:1122:3344:101::c]:42016 method = GET remote_addr = [fd00:1122:3344:101::d]:37550 req_id = 27dd1bd0-1d50-46c6-9066-100e4c0a19e4 response_code = 200 uri = /5d0959e5-47a6-4bf3-a52b-57950b9d900d
36942025-08-26T21:48:17.848ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 28 reason = Timeout
36952025-08-26T21:48:17.848ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
36962025-08-26T21:48:17.849ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
36972025-08-26T21:48:17.850ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36982025-08-26T21:48:17.853ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
36992025-08-26T21:48:17.853ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
37002025-08-26T21:48:17.853ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
37012025-08-26T21:48:17.855ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
37022025-08-26T21:48:17.855ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 6.706701ms iteration = 28 status = {"chicken_switches_updated":false}
37032025-08-26T21:48:22.848ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 29 reason = Timeout
37042025-08-26T21:48:22.848ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
37052025-08-26T21:48:22.849ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
37062025-08-26T21:48:22.850ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37072025-08-26T21:48:22.853ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
37082025-08-26T21:48:22.853ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
37092025-08-26T21:48:22.854ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
37102025-08-26T21:48:22.855ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
37112025-08-26T21:48:22.855ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.067899ms iteration = 29 status = {"chicken_switches_updated":false}
37122025-08-26T21:48:22.861ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 15 reason = Timeout
37132025-08-26T21:48:22.861ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
37142025-08-26T21:48:22.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
37152025-08-26T21:48:22.863ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
37162025-08-26T21:48:22.864ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
37172025-08-26T21:48:22.866ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.225795ms iteration = 15 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
37182025-08-26T21:48:22.873ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 15 reason = Timeout
37192025-08-26T21:48:22.873ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
37202025-08-26T21:48:22.873ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
37212025-08-26T21:48:22.875ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37222025-08-26T21:48:22.878ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
37232025-08-26T21:48:22.878ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
37242025-08-26T21:48:22.878ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
37252025-08-26T21:48:22.879ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
37262025-08-26T21:48:22.879ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
37272025-08-26T21:48:22.880ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37282025-08-26T21:48:22.883ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
37292025-08-26T21:48:22.883ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
37302025-08-26T21:48:22.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
37312025-08-26T21:48:22.899ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
37322025-08-26T21:48:22.899ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 26.462107ms iteration = 15 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
37332025-08-26T21:48:25.353ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 280 local_addr = [fd00:1122:3344:101::c]:42016 method = GET remote_addr = [fd00:1122:3344:101::d]:37550 req_id = 2d8a1892-1399-40aa-b207-5c7def5ad93a response_code = 200 uri = /5d0959e5-47a6-4bf3-a52b-57950b9d900d
37342025-08-26T21:48:27.848ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 30 reason = Timeout
37352025-08-26T21:48:27.848ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
37362025-08-26T21:48:27.849ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
37372025-08-26T21:48:27.850ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37382025-08-26T21:48:27.853ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
37392025-08-26T21:48:27.853ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
37402025-08-26T21:48:27.854ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
37412025-08-26T21:48:27.855ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
37422025-08-26T21:48:27.855ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.282725ms iteration = 30 status = {"chicken_switches_updated":false}
37432025-08-26T21:48:30.346ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:1025 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::d]:35905
37442025-08-26T21:48:30.347ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:35905 req_id = 0f74fe65-4585-4034-8c76-841318b7ed12 roles = RoleSet { roles: {} } uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?sort_by=id_ascending
37452025-08-26T21:48:30.347ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:35905 req_id = 0f74fe65-4585-4034-8c76-841318b7ed12 resource = Database result = Ok(()) uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?sort_by=id_ascending
37462025-08-26T21:48:30.350ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 3630 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:35905 req_id = 0f74fe65-4585-4034-8c76-841318b7ed12 response_code = 200 uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?sort_by=id_ascending
37472025-08-26T21:48:30.350ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:35905 req_id = 5ac811a6-1e58-4e36-a809-bc8807f9329c roles = RoleSet { roles: {} } uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMzhjYTg4YS1lZGE1LTQwZGEtOGE4MS1hN2UzMzdhMmM4ZmIifX0%3D
37482025-08-26T21:48:30.351ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:35905 req_id = 5ac811a6-1e58-4e36-a809-bc8807f9329c resource = Database result = Ok(()) uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMzhjYTg4YS1lZGE1LTQwZGEtOGE4MS1hN2UzMzdhMmM4ZmIifX0%3D
37492025-08-26T21:48:30.353ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 2482 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:35905 req_id = 5ac811a6-1e58-4e36-a809-bc8807f9329c response_code = 200 uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMzhjYTg4YS1lZGE1LTQwZGEtOGE4MS1hN2UzMzdhMmM4ZmIifX0%3D
37502025-08-26T21:48:32.834ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 6 reason = Timeout
37512025-08-26T21:48:32.834ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 6 reason = Timeout
37522025-08-26T21:48:32.834ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 6 reason = Timeout
37532025-08-26T21:48:32.834ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
37542025-08-26T21:48:32.834ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
37552025-08-26T21:48:32.835ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
37562025-08-26T21:48:32.835ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
37572025-08-26T21:48:32.837ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1016, tv_nsec: 694424049 } })
37582025-08-26T21:48:32.837ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1016, tv_nsec: 694612473 } })
37592025-08-26T21:48:32.837ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1205
37602025-08-26T21:48:32.837ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1205
37612025-08-26T21:48:32.838ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.708194ms iteration = 6 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
37622025-08-26T21:48:32.848ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 31 reason = Timeout
37632025-08-26T21:48:32.848ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
37642025-08-26T21:48:32.848ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
37652025-08-26T21:48:32.850ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37662025-08-26T21:48:32.853ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
37672025-08-26T21:48:32.853ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
37682025-08-26T21:48:32.853ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
37692025-08-26T21:48:32.855ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
37702025-08-26T21:48:32.855ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 6.766983ms iteration = 31 status = {"chicken_switches_updated":false}
37712025-08-26T21:48:32.861ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 6 reason = Timeout
37722025-08-26T21:48:32.861ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 6 reason = Timeout
37732025-08-26T21:48:32.861ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
37742025-08-26T21:48:32.861ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
37752025-08-26T21:48:32.861ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 6 reason = Timeout
37762025-08-26T21:48:32.861ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
37772025-08-26T21:48:32.861ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 6 reason = Timeout
37782025-08-26T21:48:32.861ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
37792025-08-26T21:48:32.861ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
37802025-08-26T21:48:32.861ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 6 reason = Timeout
37812025-08-26T21:48:32.861ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
37822025-08-26T21:48:32.861ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 6 reason = Timeout
37832025-08-26T21:48:32.861ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 6 reason = Timeout
37842025-08-26T21:48:32.861ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
37852025-08-26T21:48:32.861ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
37862025-08-26T21:48:32.861ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
37872025-08-26T21:48:32.861ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 16 reason = Timeout
37882025-08-26T21:48:32.861ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
37892025-08-26T21:48:32.861ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 7 reason = Timeout
37902025-08-26T21:48:32.861ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
37912025-08-26T21:48:32.861ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 6 reason = Timeout
37922025-08-26T21:48:32.861ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
37932025-08-26T21:48:32.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
37942025-08-26T21:48:32.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
37952025-08-26T21:48:32.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
37962025-08-26T21:48:32.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
37972025-08-26T21:48:32.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
37982025-08-26T21:48:32.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
37992025-08-26T21:48:32.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
38002025-08-26T21:48:32.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
38012025-08-26T21:48:32.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
38022025-08-26T21:48:32.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
38032025-08-26T21:48:32.862ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 6 reason = Timeout
38042025-08-26T21:48:32.862ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 6 reason = Timeout
38052025-08-26T21:48:32.862ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 13.83\u{b5}s iteration = 6 status = {"disabled":true,"errors":[],"sps":[]}
38062025-08-26T21:48:32.862ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 6 reason = Timeout
38072025-08-26T21:48:32.862ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
38082025-08-26T21:48:32.862ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 6 reason = Timeout
38092025-08-26T21:48:32.862ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
38102025-08-26T21:48:32.862ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
38112025-08-26T21:48:32.862ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 6 reason = Timeout
38122025-08-26T21:48:32.862ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
38132025-08-26T21:48:32.862ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 6 reason = Timeout
38142025-08-26T21:48:32.862ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
38152025-08-26T21:48:32.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
38162025-08-26T21:48:32.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
38172025-08-26T21:48:32.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
38182025-08-26T21:48:32.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
38192025-08-26T21:48:32.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
38202025-08-26T21:48:32.863ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38212025-08-26T21:48:32.863ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38222025-08-26T21:48:32.863ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38232025-08-26T21:48:32.863ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
38242025-08-26T21:48:32.863ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38252025-08-26T21:48:32.863ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
38262025-08-26T21:48:32.863ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
38272025-08-26T21:48:32.863ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38282025-08-26T21:48:32.863ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
38292025-08-26T21:48:32.863ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
38302025-08-26T21:48:32.864ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38312025-08-26T21:48:32.864ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
38322025-08-26T21:48:32.864ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
38332025-08-26T21:48:32.864ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 2.294132ms iteration = 6 status = {"errors":[],"finish_invoked_ok":[]}
38342025-08-26T21:48:32.865ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
38352025-08-26T21:48:32.865ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 2.463375ms iteration = 6 status = {"errors":[],"garbage_collect_requested":[]}
38362025-08-26T21:48:32.866ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.735276ms iteration = 16 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
38372025-08-26T21:48:32.866ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
38382025-08-26T21:48:32.866ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
38392025-08-26T21:48:32.866ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
38402025-08-26T21:48:32.866ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
38412025-08-26T21:48:32.866ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
38422025-08-26T21:48:32.866ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
38432025-08-26T21:48:32.867ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
38442025-08-26T21:48:32.867ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
38452025-08-26T21:48:32.867ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
38462025-08-26T21:48:32.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
38472025-08-26T21:48:32.868ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
38482025-08-26T21:48:32.868ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
38492025-08-26T21:48:32.868ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 5.821251ms iteration = 6 status = {"errors":[],"requests_created_ok":[]}
38502025-08-26T21:48:32.868ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
38512025-08-26T21:48:32.868ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
38522025-08-26T21:48:32.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
38532025-08-26T21:48:32.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
38542025-08-26T21:48:32.868ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
38552025-08-26T21:48:32.868ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
38562025-08-26T21:48:32.869ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
38572025-08-26T21:48:32.869ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
38582025-08-26T21:48:32.869ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
38592025-08-26T21:48:32.869ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
38602025-08-26T21:48:32.869ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
38612025-08-26T21:48:32.869ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
38622025-08-26T21:48:32.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
38632025-08-26T21:48:32.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
38642025-08-26T21:48:32.869ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 7.931429ms iteration = 6 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
38652025-08-26T21:48:32.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
38662025-08-26T21:48:32.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
38672025-08-26T21:48:32.870ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 7.488079ms iteration = 6 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
38682025-08-26T21:48:32.870ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38692025-08-26T21:48:32.872ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1016, tv_nsec: 729638213 } })
38702025-08-26T21:48:32.872ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1205 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
38712025-08-26T21:48:32.873ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
38722025-08-26T21:48:32.873ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
38732025-08-26T21:48:32.873ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
38742025-08-26T21:48:32.873ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
38752025-08-26T21:48:32.874ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
38762025-08-26T21:48:32.874ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
38772025-08-26T21:48:32.875ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 16 reason = Timeout
38782025-08-26T21:48:32.875ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
38792025-08-26T21:48:32.875ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:550 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
38802025-08-26T21:48:32.875ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 13.962805ms iteration = 6 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
38812025-08-26T21:48:32.875ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
38822025-08-26T21:48:32.875ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
38832025-08-26T21:48:32.876ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38842025-08-26T21:48:32.881ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
38852025-08-26T21:48:32.881ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
38862025-08-26T21:48:32.881ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
38872025-08-26T21:48:32.885ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
38882025-08-26T21:48:32.885ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
38892025-08-26T21:48:32.887ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38902025-08-26T21:48:32.892ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
38912025-08-26T21:48:32.892ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
38922025-08-26T21:48:32.892ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
38932025-08-26T21:48:32.908ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
38942025-08-26T21:48:32.908ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 33.057225ms iteration = 16 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
38952025-08-26T21:48:32.913ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1213 zone_address = fd00:1122:3344:101::2
38962025-08-26T21:48:32.913ZDEBGnexus (MgsClient): client request background_task = nat_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
38972025-08-26T21:48:32.913ZDEBGnexus (MgsClient): client response background_task = nat_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "4df4b1a7-cdcb-4f0c-8427-c744ae81bc3f", "content-length": "26", "date": "Tue, 26 Aug 2025 21:48:32 GMT"} })
38982025-08-26T21:48:32.913ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1216 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
38992025-08-26T21:48:32.913ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1213 zone_address = fd00:1122:3344:101::2
39002025-08-26T21:48:32.913ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
39012025-08-26T21:48:32.914ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "747f100d-f89f-4893-b95b-52c3aa152de7", "content-length": "26", "date": "Tue, 26 Aug 2025 21:48:32 GMT"} })
39022025-08-26T21:48:32.914ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1216 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
39032025-08-26T21:48:32.915ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
39042025-08-26T21:48:32.915ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
39052025-08-26T21:48:32.917ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39062025-08-26T21:48:32.921ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
39072025-08-26T21:48:32.921ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
39082025-08-26T21:48:32.921ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 59.369648ms iteration = 6 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
39092025-08-26T21:48:32.922ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
39102025-08-26T21:48:32.925ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
39112025-08-26T21:48:32.925ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
39122025-08-26T21:48:32.925ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
39132025-08-26T21:48:32.925ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
39142025-08-26T21:48:32.925ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 64.667968ms iteration = 6 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
39152025-08-26T21:48:32.930ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39162025-08-26T21:48:32.932ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
39172025-08-26T21:48:32.932ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
39182025-08-26T21:48:32.932ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
39192025-08-26T21:48:32.934ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:407
39202025-08-26T21:48:32.934ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 72.979025ms iteration = 6 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
39212025-08-26T21:48:32.936ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
39222025-08-26T21:48:32.936ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
39232025-08-26T21:48:32.938ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
39242025-08-26T21:48:32.939ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
39252025-08-26T21:48:32.940ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39262025-08-26T21:48:32.943ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: dd0c655a-95c2-48fc-b4c3-9733ed4b0faa, lookup_type: ById(dd0c655a-95c2-48fc-b4c3-9733ed4b0faa) } result = Ok(())
39272025-08-26T21:48:32.943ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
39282025-08-26T21:48:32.944ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
39292025-08-26T21:48:32.945ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 84.667068ms iteration = 6 status = 0
39302025-08-26T21:48:32.964ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 103.097114ms iteration = 7 status = {"physical_disks_added":0}
39312025-08-26T21:48:32.968ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
39322025-08-26T21:48:32.968ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
39332025-08-26T21:48:32.977ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
39342025-08-26T21:48:32.977ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
39352025-08-26T21:48:32.978ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39362025-08-26T21:48:32.980ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
39372025-08-26T21:48:32.980ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
39382025-08-26T21:48:32.980ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
39392025-08-26T21:48:32.982ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39402025-08-26T21:48:32.984ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
39412025-08-26T21:48:32.984ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
39422025-08-26T21:48:32.984ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
39432025-08-26T21:48:33.017ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 182.571606ms iteration = 6 status = {}
39442025-08-26T21:48:33.026ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1213 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 zone_address = fd00:1122:3344:101::2
39452025-08-26T21:48:33.026ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
39462025-08-26T21:48:33.027ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "c1ad9169-11e9-4be5-a565-6accd26dbf59", "content-length": "26", "date": "Tue, 26 Aug 2025 21:48:32 GMT"} })
39472025-08-26T21:48:33.027ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1216 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
39482025-08-26T21:48:33.028ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
39492025-08-26T21:48:33.060ZINFOnexus (BackgroundTasks): VPC route manager sled dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
39502025-08-26T21:48:33.060ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
39512025-08-26T21:48:33.060ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "8c2e0684-d647-48e1-b0b4-c9aa2f70063a", "content-length": "372", "date": "Tue, 26 Aug 2025 21:48:33 GMT"} })
39522025-08-26T21:48:33.060ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
39532025-08-26T21:48:33.061ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
39542025-08-26T21:48:33.066ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
39552025-08-26T21:48:33.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
39562025-08-26T21:48:33.068ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 233.30797ms iteration = 6 status = null
39572025-08-26T21:48:33.074ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
39582025-08-26T21:48:33.074ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
39592025-08-26T21:48:33.075ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
39602025-08-26T21:48:33.075ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "9837d64f-6e32-4df2-bab0-1df3fd72b594", "date": "Tue, 26 Aug 2025 21:48:32 GMT"} })
39612025-08-26T21:48:33.075ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
39622025-08-26T21:48:33.075ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "ad97d3a5-a872-48c0-948d-9b019ffc78f8", "content-length": "691", "date": "Tue, 26 Aug 2025 21:48:32 GMT"} })
39632025-08-26T21:48:33.075ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
39642025-08-26T21:48:33.075ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
39652025-08-26T21:48:33.075ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
39662025-08-26T21:48:33.075ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
39672025-08-26T21:48:33.077ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 216.184343ms iteration = 6 status = {}
39682025-08-26T21:48:33.077ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
39692025-08-26T21:48:33.077ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
39702025-08-26T21:48:33.080ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
39712025-08-26T21:48:33.081ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
39722025-08-26T21:48:33.084ZINFOnexus (BackgroundTasks): VPC route manager sled dd0c655a-95c2-48fc-b4c3-9733ed4b0faa push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
39732025-08-26T21:48:33.084ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
39742025-08-26T21:48:33.084ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "89ca4950-8419-4f96-965b-09933d6208e7", "date": "Tue, 26 Aug 2025 21:48:33 GMT"} })
39752025-08-26T21:48:33.085ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 224.596203ms iteration = 6 status = {}
39762025-08-26T21:48:33.146ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
39772025-08-26T21:48:33.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
39782025-08-26T21:48:33.148ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: d24c5c55-3a21-4a91-aeb4-45362e822df6, rack_id: e37a46f2-f789-4ca4-9841-f8c6dd705964, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(88d84f37-a4c8-4abc-af1f-27987b0d39d9) } rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0
39792025-08-26T21:48:33.148ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
39802025-08-26T21:48:33.148ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
39812025-08-26T21:48:33.159ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1832 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
39822025-08-26T21:48:33.159ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1859 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 88d84f37-a4c8-4abc-af1f-27987b0d39d9
39832025-08-26T21:48:33.159ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
39842025-08-26T21:48:33.159ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
39852025-08-26T21:48:33.159ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
39862025-08-26T21:48:33.159ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
39872025-08-26T21:48:33.159ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
39882025-08-26T21:48:33.159ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2006 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
39892025-08-26T21:48:33.159ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:408 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
39902025-08-26T21:48:33.159ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2042 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
39912025-08-26T21:48:33.159ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
39922025-08-26T21:48:33.159ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
39932025-08-26T21:48:33.160ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
39942025-08-26T21:48:33.162ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0
39952025-08-26T21:48:33.162ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
39962025-08-26T21:48:33.267ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "2df29b38-21f9-4a0e-a0f6-f265e45f7286", "date": "Tue, 26 Aug 2025 21:48:33 GMT"} })
39972025-08-26T21:48:33.267ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch = Switch0
39982025-08-26T21:48:33.267ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
39992025-08-26T21:48:33.267ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "69f681b1-fc96-40aa-af01-95fdb3e5afb2", "content-length": "682", "date": "Tue, 26 Aug 2025 21:48:33 GMT"} })
40002025-08-26T21:48:33.267ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
40012025-08-26T21:48:33.267ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
40022025-08-26T21:48:33.271ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
40032025-08-26T21:48:33.271ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
40042025-08-26T21:48:33.275ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
40052025-08-26T21:48:33.275ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
40062025-08-26T21:48:33.277ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
40072025-08-26T21:48:33.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
40082025-08-26T21:48:33.284ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 423.70048ms iteration = 6 status = {}
40092025-08-26T21:48:35.354ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 369 local_addr = [fd00:1122:3344:101::c]:42016 method = GET remote_addr = [fd00:1122:3344:101::d]:37550 req_id = 3eac5ad0-197f-4363-8aed-6e8d7e5beaea response_code = 200 uri = /5d0959e5-47a6-4bf3-a52b-57950b9d900d
40102025-08-26T21:48:37.848ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 32 reason = Timeout
40112025-08-26T21:48:37.848ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
40122025-08-26T21:48:37.848ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
40132025-08-26T21:48:37.850ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40142025-08-26T21:48:37.853ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
40152025-08-26T21:48:37.853ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
40162025-08-26T21:48:37.853ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
40172025-08-26T21:48:37.855ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
40182025-08-26T21:48:37.855ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.789976ms iteration = 32 status = {"chicken_switches_updated":false}
40192025-08-26T21:48:42.848ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 33 reason = Timeout
40202025-08-26T21:48:42.848ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
40212025-08-26T21:48:42.848ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
40222025-08-26T21:48:42.850ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40232025-08-26T21:48:42.853ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
40242025-08-26T21:48:42.853ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
40252025-08-26T21:48:42.853ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
40262025-08-26T21:48:42.854ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
40272025-08-26T21:48:42.854ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 6.432295ms iteration = 33 status = {"chicken_switches_updated":false}
40282025-08-26T21:48:42.861ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 17 reason = Timeout
40292025-08-26T21:48:42.861ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
40302025-08-26T21:48:42.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
40312025-08-26T21:48:42.873ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 17 reason = Timeout
40322025-08-26T21:48:42.873ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
40332025-08-26T21:48:42.874ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
40342025-08-26T21:48:42.876ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40352025-08-26T21:48:42.879ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
40362025-08-26T21:48:42.879ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
40372025-08-26T21:48:42.879ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
40382025-08-26T21:48:42.880ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
40392025-08-26T21:48:42.880ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
40402025-08-26T21:48:42.882ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40412025-08-26T21:48:42.885ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
40422025-08-26T21:48:42.885ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
40432025-08-26T21:48:42.885ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
40442025-08-26T21:48:42.904ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
40452025-08-26T21:48:42.904ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 30.522848ms iteration = 17 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
40462025-08-26T21:48:42.909ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
40472025-08-26T21:48:42.909ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
40482025-08-26T21:48:42.910ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 49.032016ms iteration = 17 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
40492025-08-26T21:48:45.353ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 320 local_addr = [fd00:1122:3344:101::c]:42016 method = GET remote_addr = [fd00:1122:3344:101::d]:37550 req_id = 63ef5d93-5b23-47ba-911c-92a55e96f48d response_code = 200 uri = /5d0959e5-47a6-4bf3-a52b-57950b9d900d
40502025-08-26T21:48:47.848ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 34 reason = Timeout
40512025-08-26T21:48:47.848ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
40522025-08-26T21:48:47.849ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
40532025-08-26T21:48:47.850ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40542025-08-26T21:48:47.853ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
40552025-08-26T21:48:47.853ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
40562025-08-26T21:48:47.853ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
40572025-08-26T21:48:47.855ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
40582025-08-26T21:48:47.855ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.422467ms iteration = 34 status = {"chicken_switches_updated":false}
40592025-08-26T21:48:52.847ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 35 reason = Timeout
40602025-08-26T21:48:52.848ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
40612025-08-26T21:48:52.848ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
40622025-08-26T21:48:52.850ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40632025-08-26T21:48:52.852ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
40642025-08-26T21:48:52.852ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
40652025-08-26T21:48:52.853ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
40662025-08-26T21:48:52.854ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
40672025-08-26T21:48:52.854ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 6.544907ms iteration = 35 status = {"chicken_switches_updated":false}
40682025-08-26T21:48:52.861ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 18 reason = Timeout
40692025-08-26T21:48:52.861ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
40702025-08-26T21:48:52.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
40712025-08-26T21:48:52.863ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
40722025-08-26T21:48:52.863ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
40732025-08-26T21:48:52.865ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.580663ms iteration = 18 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
40742025-08-26T21:48:52.873ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 18 reason = Timeout
40752025-08-26T21:48:52.873ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
40762025-08-26T21:48:52.873ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
40772025-08-26T21:48:52.875ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40782025-08-26T21:48:52.878ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
40792025-08-26T21:48:52.878ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
40802025-08-26T21:48:52.878ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
40812025-08-26T21:48:52.880ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
40822025-08-26T21:48:52.880ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
40832025-08-26T21:48:52.882ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40842025-08-26T21:48:52.885ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
40852025-08-26T21:48:52.885ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
40862025-08-26T21:48:52.885ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
40872025-08-26T21:48:52.902ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
40882025-08-26T21:48:52.902ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 28.475022ms iteration = 18 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
40892025-08-26T21:48:55.354ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 513 local_addr = [fd00:1122:3344:101::c]:42016 method = GET remote_addr = [fd00:1122:3344:101::d]:37550 req_id = a87915ab-bc40-404c-9291-deeb3493cce6 response_code = 200 uri = /5d0959e5-47a6-4bf3-a52b-57950b9d900d
40902025-08-26T21:48:57.848ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 36 reason = Timeout
40912025-08-26T21:48:57.848ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
40922025-08-26T21:48:57.848ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
40932025-08-26T21:48:57.850ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40942025-08-26T21:48:57.853ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
40952025-08-26T21:48:57.853ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
40962025-08-26T21:48:57.853ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
40972025-08-26T21:48:57.855ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
40982025-08-26T21:48:57.855ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.314205ms iteration = 36 status = {"chicken_switches_updated":false}
40992025-08-26T21:49:02.833ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 4 reason = Timeout
41002025-08-26T21:49:02.834ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 4 reason = Timeout
41012025-08-26T21:49:02.834ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 4 reason = Timeout
41022025-08-26T21:49:02.834ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-08-26 21:39:02.833895925 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
41032025-08-26T21:49:02.834ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
41042025-08-26T21:49:02.834ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
41052025-08-26T21:49:02.834ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 7 reason = Timeout
41062025-08-26T21:49:02.834ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 4 reason = Timeout
41072025-08-26T21:49:02.834ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
41082025-08-26T21:49:02.834ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 7 reason = Timeout
41092025-08-26T21:49:02.834ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
41102025-08-26T21:49:02.834ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
41112025-08-26T21:49:02.834ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 7 reason = Timeout
41122025-08-26T21:49:02.834ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
41132025-08-26T21:49:02.834ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 6 reason = Timeout
41142025-08-26T21:49:02.834ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 6 reason = Timeout
41152025-08-26T21:49:02.834ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 4 reason = Timeout
41162025-08-26T21:49:02.834ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
41172025-08-26T21:49:02.834ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
41182025-08-26T21:49:02.834ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 4 reason = Timeout
41192025-08-26T21:49:02.834ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
41202025-08-26T21:49:02.834ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
41212025-08-26T21:49:02.834ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
41222025-08-26T21:49:02.834ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
41232025-08-26T21:49:02.834ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
41242025-08-26T21:49:02.834ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
41252025-08-26T21:49:02.834ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("852fd1a3-de9a-47e8-a667-5e8dab880ad3.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("c68f1501-294c-4874-92ca-8fc0546a5870.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("d741a72f-8745-4723-9e80-e1e72987266b.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("d741a72f-8745-4723-9e80-e1e72987266b.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:3::1)) }], valid_until: Instant { tv_sec: 1046, tv_nsec: 692132377 } })
41262025-08-26T21:49:02.834ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("60abf731-31e1-4111-a3e2-65da89eaed87.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("84c849eb-0949-41d9-ad02-9a5509144564.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("84c849eb-0949-41d9-ad02-9a5509144564.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::8)) }], valid_until: Instant { tv_sec: 1046, tv_nsec: 692138217 } })
41272025-08-26T21:49:02.835ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
41282025-08-26T21:49:02.835ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } dns_group = internal
41292025-08-26T21:49:02.835ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
41302025-08-26T21:49:02.835ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.127985ms iteration = 4 status = {"addresses":["[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:3::1]:5353"]}
41312025-08-26T21:49:02.835ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.353861ms iteration = 4 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
41322025-08-26T21:49:02.835ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
41332025-08-26T21:49:02.835ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
41342025-08-26T21:49:02.835ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
41352025-08-26T21:49:02.836ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41362025-08-26T21:49:02.836ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1046, tv_nsec: 694328537 } })
41372025-08-26T21:49:02.836ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 5 reason = Timeout
41382025-08-26T21:49:02.836ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:91 reason = blueprint disabled target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8
41392025-08-26T21:49:02.836ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 13.231\u{b5}s iteration = 5 status = {"enabled":false,"target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8"}
41402025-08-26T21:49:02.837ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.14217ms iteration = 7 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
41412025-08-26T21:49:02.837ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1205
41422025-08-26T21:49:02.837ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-08-26 21:39:02.833895925 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
41432025-08-26T21:49:02.837ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 3.348785ms iteration = 4 status = {"expiration":"2025-08-26T21:39:02.833895925Z","pruned":[]}
41442025-08-26T21:49:02.837ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1046, tv_nsec: 694797117 } })
41452025-08-26T21:49:02.837ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41462025-08-26T21:49:02.837ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41472025-08-26T21:49:02.837ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1205
41482025-08-26T21:49:02.838ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
41492025-08-26T21:49:02.838ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
41502025-08-26T21:49:02.839ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
41512025-08-26T21:49:02.840ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
41522025-08-26T21:49:02.840ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
41532025-08-26T21:49:02.841ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
41542025-08-26T21:49:02.841ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
41552025-08-26T21:49:02.841ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41562025-08-26T21:49:02.844ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
41572025-08-26T21:49:02.844ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
41582025-08-26T21:49:02.844ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
41592025-08-26T21:49:02.847ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 37 reason = Timeout
41602025-08-26T21:49:02.847ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
41612025-08-26T21:49:02.848ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
41622025-08-26T21:49:02.849ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41632025-08-26T21:49:02.852ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
41642025-08-26T21:49:02.852ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
41652025-08-26T21:49:02.852ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
41662025-08-26T21:49:02.854ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
41672025-08-26T21:49:02.854ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 6.826223ms iteration = 37 status = {"chicken_switches_updated":false}
41682025-08-26T21:49:02.861ZDEBGnexus (BackgroundTasks): activating background_task = crdb_node_id_collector iteration = 3 reason = Timeout
41692025-08-26T21:49:02.861ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 7 reason = Timeout
41702025-08-26T21:49:02.861ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
41712025-08-26T21:49:02.861ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
41722025-08-26T21:49:02.861ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
41732025-08-26T21:49:02.861ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 7 reason = Timeout
41742025-08-26T21:49:02.861ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
41752025-08-26T21:49:02.861ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 7 reason = Timeout
41762025-08-26T21:49:02.861ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
41772025-08-26T21:49:02.861ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 4 reason = Timeout
41782025-08-26T21:49:02.861ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:53
41792025-08-26T21:49:02.861ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
41802025-08-26T21:49:02.861ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 7 reason = Timeout
41812025-08-26T21:49:02.861ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
41822025-08-26T21:49:02.861ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 7 reason = Timeout
41832025-08-26T21:49:02.861ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
41842025-08-26T21:49:02.861ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 19 reason = Timeout
41852025-08-26T21:49:02.861ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
41862025-08-26T21:49:02.861ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 6 reason = Timeout
41872025-08-26T21:49:02.861ZDEBGnexus (BackgroundTasks): blueprint planning disabled, doing nothing background_task = blueprint_planner
41882025-08-26T21:49:02.861ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 8.24\u{b5}s iteration = 6 status = "Disabled"
41892025-08-26T21:49:02.861ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 7 reason = Timeout
41902025-08-26T21:49:02.861ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 4 reason = Timeout
41912025-08-26T21:49:02.861ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
41922025-08-26T21:49:02.861ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
41932025-08-26T21:49:02.861ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 4 reason = Timeout
41942025-08-26T21:49:02.861ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 7 reason = Timeout
41952025-08-26T21:49:02.861ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
41962025-08-26T21:49:02.861ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 7 reason = Timeout
41972025-08-26T21:49:02.861ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
41982025-08-26T21:49:02.861ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
41992025-08-26T21:49:02.861ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
42002025-08-26T21:49:02.861ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 8 reason = Timeout
42012025-08-26T21:49:02.861ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
42022025-08-26T21:49:02.861ZDEBGnexus (BackgroundTasks): activating background_task = webhook_deliverator iteration = 4 reason = Timeout
42032025-08-26T21:49:02.862ZDEBGnexus (BackgroundTasks): roles background_task = webhook_deliverator roles = RoleSet { roles: {} }
42042025-08-26T21:49:02.862ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 7 reason = Timeout
42052025-08-26T21:49:02.862ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
42062025-08-26T21:49:02.862ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 7 reason = Timeout
42072025-08-26T21:49:02.862ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
42082025-08-26T21:49:02.862ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 7 reason = Timeout
42092025-08-26T21:49:02.862ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
42102025-08-26T21:49:02.862ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 7 reason = Timeout
42112025-08-26T21:49:02.862ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
42122025-08-26T21:49:02.862ZDEBGnexus (BackgroundTasks): activating background_task = alert_dispatcher iteration = 4 reason = Timeout
42132025-08-26T21:49:02.862ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
42142025-08-26T21:49:02.862ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 7 reason = Timeout
42152025-08-26T21:49:02.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
42162025-08-26T21:49:02.862ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 7 reason = Timeout
42172025-08-26T21:49:02.862ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 18.28\u{b5}s iteration = 7 status = {"disabled":true,"errors":[],"sps":[]}
42182025-08-26T21:49:02.862ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
42192025-08-26T21:49:02.862ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 4 reason = Timeout
42202025-08-26T21:49:02.862ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
42212025-08-26T21:49:02.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
42222025-08-26T21:49:02.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
42232025-08-26T21:49:02.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
42242025-08-26T21:49:02.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
42252025-08-26T21:49:02.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
42262025-08-26T21:49:02.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
42272025-08-26T21:49:02.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
42282025-08-26T21:49:02.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
42292025-08-26T21:49:02.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
42302025-08-26T21:49:02.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
42312025-08-26T21:49:02.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
42322025-08-26T21:49:02.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
42332025-08-26T21:49:02.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
42342025-08-26T21:49:02.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
42352025-08-26T21:49:02.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
42362025-08-26T21:49:02.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
42372025-08-26T21:49:02.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = webhook_deliverator resource = Database result = Ok(())
42382025-08-26T21:49:02.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = alert_dispatcher resource = Database result = Ok(())
42392025-08-26T21:49:02.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
42402025-08-26T21:49:02.862ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
42412025-08-26T21:49:02.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
42422025-08-26T21:49:02.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
42432025-08-26T21:49:02.863ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
42442025-08-26T21:49:02.863ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
42452025-08-26T21:49:02.863ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
42462025-08-26T21:49:02.863ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
42472025-08-26T21:49:02.863ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
42482025-08-26T21:49:02.863ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
42492025-08-26T21:49:02.864ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
42502025-08-26T21:49:02.865ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42512025-08-26T21:49:02.866ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42522025-08-26T21:49:02.867ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
42532025-08-26T21:49:02.867ZINFOnexus (BackgroundTasks): all webhook delivery tasks completed background_task = webhook_deliverator file = nexus/src/app/background/tasks/webhook_deliverator.rs:204 num_receivers = 0
42542025-08-26T21:49:02.867ZDEBGnexus (BackgroundTasks): activation complete background_task = webhook_deliverator elapsed = 5.877842ms iteration = 4 status = {"by_rx":{},"error":null}
42552025-08-26T21:49:02.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
42562025-08-26T21:49:02.868ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42572025-08-26T21:49:02.868ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
42582025-08-26T21:49:02.868ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
42592025-08-26T21:49:02.868ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
42602025-08-26T21:49:02.868ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
42612025-08-26T21:49:02.868ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:170 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
42622025-08-26T21:49:02.868ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 6.674971ms iteration = 4 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
42632025-08-26T21:49:02.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
42642025-08-26T21:49:02.869ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
42652025-08-26T21:49:02.869ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
42662025-08-26T21:49:02.869ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42672025-08-26T21:49:02.869ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
42682025-08-26T21:49:02.869ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
42692025-08-26T21:49:02.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
42702025-08-26T21:49:02.870ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 8.349739ms iteration = 19 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
42712025-08-26T21:49:02.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
42722025-08-26T21:49:02.870ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 8.626825ms iteration = 4 status = {"errors":[],"found_port_ok":[]}
42732025-08-26T21:49:02.871ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
42742025-08-26T21:49:02.871ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
42752025-08-26T21:49:02.871ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
42762025-08-26T21:49:02.871ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 9.483164ms iteration = 7 status = {"errors":[],"requests_created_ok":[]}
42772025-08-26T21:49:02.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
42782025-08-26T21:49:02.871ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42792025-08-26T21:49:02.872ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
42802025-08-26T21:49:02.872ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
42812025-08-26T21:49:02.872ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
42822025-08-26T21:49:02.872ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
42832025-08-26T21:49:02.872ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
42842025-08-26T21:49:02.873ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 19 reason = Timeout
42852025-08-26T21:49:02.873ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
42862025-08-26T21:49:02.873ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
42872025-08-26T21:49:02.873ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42882025-08-26T21:49:02.873ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42892025-08-26T21:49:02.873ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
42902025-08-26T21:49:02.875ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
42912025-08-26T21:49:02.876ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = alert_dispatcher resource = Database result = Ok(())
42922025-08-26T21:49:02.876ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
42932025-08-26T21:49:02.876ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
42942025-08-26T21:49:02.876ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
42952025-08-26T21:49:02.877ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42962025-08-26T21:49:02.877ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
42972025-08-26T21:49:02.877ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:138 found = 0 not_ready_to_be_deleted = 0
42982025-08-26T21:49:02.877ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 15.663213ms iteration = 4 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
42992025-08-26T21:49:02.877ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
43002025-08-26T21:49:02.877ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
43012025-08-26T21:49:02.877ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
43022025-08-26T21:49:02.878ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
43032025-08-26T21:49:02.878ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
43042025-08-26T21:49:02.878ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
43052025-08-26T21:49:02.878ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
43062025-08-26T21:49:02.878ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
43072025-08-26T21:49:02.879ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43082025-08-26T21:49:02.880ZDEBGnexus (BackgroundTasks): activation complete background_task = alert_dispatcher elapsed = 18.129759ms iteration = 4 status = {"dispatched":[],"errors":[],"glob_version":"183.0.0","globs_reprocessed":{},"no_receivers":[]}
43092025-08-26T21:49:02.880ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
43102025-08-26T21:49:02.880ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
43112025-08-26T21:49:02.881ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
43122025-08-26T21:49:02.881ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43132025-08-26T21:49:02.883ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
43142025-08-26T21:49:02.883ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
43152025-08-26T21:49:02.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
43162025-08-26T21:49:02.883ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43172025-08-26T21:49:02.884ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
43182025-08-26T21:49:02.884ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
43192025-08-26T21:49:02.885ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
43202025-08-26T21:49:02.886ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
43212025-08-26T21:49:02.886ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
43222025-08-26T21:49:02.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
43232025-08-26T21:49:02.888ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
43242025-08-26T21:49:02.888ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
43252025-08-26T21:49:02.891ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
43262025-08-26T21:49:02.891ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
43272025-08-26T21:49:02.891ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
43282025-08-26T21:49:02.891ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
43292025-08-26T21:49:02.891ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
43302025-08-26T21:49:02.895ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
43312025-08-26T21:49:02.895ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
43322025-08-26T21:49:02.896ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43332025-08-26T21:49:02.896ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43342025-08-26T21:49:02.897ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43352025-08-26T21:49:02.901ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
43362025-08-26T21:49:02.902ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
43372025-08-26T21:49:02.902ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
43382025-08-26T21:49:02.902ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
43392025-08-26T21:49:02.902ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
43402025-08-26T21:49:02.902ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
43412025-08-26T21:49:02.902ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
43422025-08-26T21:49:02.902ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
43432025-08-26T21:49:02.902ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
43442025-08-26T21:49:02.904ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
43452025-08-26T21:49:02.904ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 8e720f02-dde8-4423-a6e3-91698373805a dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
43462025-08-26T21:49:02.904ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
43472025-08-26T21:49:02.904ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
43482025-08-26T21:49:02.907ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
43492025-08-26T21:49:02.907ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
43502025-08-26T21:49:02.907ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
43512025-08-26T21:49:02.907ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
43522025-08-26T21:49:02.907ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
43532025-08-26T21:49:02.907ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43542025-08-26T21:49:02.908ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43552025-08-26T21:49:02.912ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
43562025-08-26T21:49:02.912ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
43572025-08-26T21:49:02.913ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
43582025-08-26T21:49:02.913ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
43592025-08-26T21:49:02.913ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
43602025-08-26T21:49:02.914ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
43612025-08-26T21:49:02.914ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
43622025-08-26T21:49:02.914ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = dfb404ab-acb8-43de-afe8-b429af0c161d dns_zone_name = oxide.test found_so_far = 0 version = 2
43632025-08-26T21:49:02.914ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
43642025-08-26T21:49:02.914ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
43652025-08-26T21:49:02.918ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 8e720f02-dde8-4423-a6e3-91698373805a dns_zone_name = control-plane.oxide.internal found_so_far = 59 version = 1
43662025-08-26T21:49:02.918ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
43672025-08-26T21:49:02.918ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2025-08-26 21:46:04.065217 UTC dns_group = internal generation = 1
43682025-08-26T21:49:02.918ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 84.385062ms iteration = 4 status = {"generation":1}
43692025-08-26T21:49:02.919ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43702025-08-26T21:49:02.920ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43712025-08-26T21:49:02.923ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
43722025-08-26T21:49:02.923ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
43732025-08-26T21:49:02.923ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
43742025-08-26T21:49:02.924ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
43752025-08-26T21:49:02.924ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
43762025-08-26T21:49:02.924ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
43772025-08-26T21:49:02.926ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
43782025-08-26T21:49:02.926ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
43792025-08-26T21:49:02.928ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 66.817526ms iteration = 7 status = {"errors":[],"garbage_collect_requested":[]}
43802025-08-26T21:49:02.930ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
43812025-08-26T21:49:02.930ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
43822025-08-26T21:49:02.931ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43832025-08-26T21:49:02.932ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = dfb404ab-acb8-43de-afe8-b429af0c161d dns_zone_name = oxide.test found_so_far = 4 version = 2
43842025-08-26T21:49:02.932ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
43852025-08-26T21:49:02.932ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
43862025-08-26T21:49:02.932ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
43872025-08-26T21:49:02.932ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2025-08-26 21:46:05.296205 UTC dns_group = external generation = 2
43882025-08-26T21:49:02.932ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 98.071261ms iteration = 4 status = {"generation":2}
43892025-08-26T21:49:02.932ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 70.427698ms iteration = 7 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
43902025-08-26T21:49:02.932ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43912025-08-26T21:49:02.933ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
43922025-08-26T21:49:02.933ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 71.41511ms iteration = 7 status = {"errors":[],"finish_invoked_ok":[]}
43932025-08-26T21:49:02.933ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
43942025-08-26T21:49:02.933ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43952025-08-26T21:49:02.934ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
43962025-08-26T21:49:02.934ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
43972025-08-26T21:49:02.934ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
43982025-08-26T21:49:02.935ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
43992025-08-26T21:49:02.935ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
44002025-08-26T21:49:02.935ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
44012025-08-26T21:49:02.935ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
44022025-08-26T21:49:02.935ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
44032025-08-26T21:49:02.936ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
44042025-08-26T21:49:02.936ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
44052025-08-26T21:49:02.936ZDEBGnexus (BackgroundTasks): activation complete background_task = crdb_node_id_collector elapsed = 75.098803ms iteration = 3 status = {"nsuccess":5}
44062025-08-26T21:49:02.936ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
44072025-08-26T21:49:02.936ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1046, tv_nsec: 794443133 } })
44082025-08-26T21:49:02.937ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 75.169164ms iteration = 7 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
44092025-08-26T21:49:02.937ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1205 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
44102025-08-26T21:49:02.938ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
44112025-08-26T21:49:02.938ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
44122025-08-26T21:49:02.939ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
44132025-08-26T21:49:02.939ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
44142025-08-26T21:49:02.939ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 66.778905ms iteration = 19 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
44152025-08-26T21:49:02.939ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
44162025-08-26T21:49:02.941ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 79.667596ms iteration = 7 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
44172025-08-26T21:49:02.944ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
44182025-08-26T21:49:02.944ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
44192025-08-26T21:49:02.946ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
44202025-08-26T21:49:02.948ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
44212025-08-26T21:49:02.948ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
44222025-08-26T21:49:02.950ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
44232025-08-26T21:49:02.951ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
44242025-08-26T21:49:02.951ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
44252025-08-26T21:49:02.951ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
44262025-08-26T21:49:02.952ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:407
44272025-08-26T21:49:02.952ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 90.924339ms iteration = 7 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
44282025-08-26T21:49:02.953ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 91.882491ms iteration = 8 status = {"physical_disks_added":0}
44292025-08-26T21:49:02.953ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
44302025-08-26T21:49:02.953ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
44312025-08-26T21:49:02.953ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
44322025-08-26T21:49:02.955ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
44332025-08-26T21:49:02.957ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
44342025-08-26T21:49:02.957ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
44352025-08-26T21:49:02.958ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
44362025-08-26T21:49:02.967ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-08-26T21:45:59.866217Z, time_modified: 2025-08-26T21:45:59.866217Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 9396aa55-de6f-4d6f-96db-d28e1c76fa9c, db_silo: Silo { identity: SiloIdentity { id: 9396aa55-de6f-4d6f-96db-d28e1c76fa9c, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-08-26T21:46:05.245521Z, time_modified: 2025-08-26T21:46:05.245521Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "e8529d720e4b0de099753dcdb373abb870af04baacd7c7dd850f5254858081af" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 9396aa55-de6f-4d6f-96db-d28e1c76fa9c, db_silo: Silo { identity: SiloIdentity { id: 9396aa55-de6f-4d6f-96db-d28e1c76fa9c, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-08-26T21:46:05.245521Z, time_modified: 2025-08-26T21:46:05.245521Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "e8529d720e4b0de099753dcdb373abb870af04baacd7c7dd850f5254858081af" }] }) }
44372025-08-26T21:49:02.967ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 133.272274ms iteration = 4 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"9396aa55-de6f-4d6f-96db-d28e1c76fa9c","tls_certs":["e8529d720e4b0de099753dcdb373abb870af04baacd7c7dd850f5254858081af"]}},"default_endpoint":{"silo_id":"9396aa55-de6f-4d6f-96db-d28e1c76fa9c","tls_certs":["e8529d720e4b0de099753dcdb373abb870af04baacd7c7dd850f5254858081af"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
44382025-08-26T21:49:02.974ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 112.415804ms iteration = 7 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
44392025-08-26T21:49:02.976ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
44402025-08-26T21:49:02.980ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
44412025-08-26T21:49:02.982ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1213 zone_address = fd00:1122:3344:101::2
44422025-08-26T21:49:02.982ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
44432025-08-26T21:49:02.982ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "3201f63e-d43f-4490-97da-0fd347e670c4", "content-length": "26", "date": "Tue, 26 Aug 2025 21:49:02 GMT"} })
44442025-08-26T21:49:02.982ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1216 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
44452025-08-26T21:49:02.984ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
44462025-08-26T21:49:02.991ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:550 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
44472025-08-26T21:49:02.991ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 129.692724ms iteration = 7 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
44482025-08-26T21:49:02.992ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1213 zone_address = fd00:1122:3344:101::2
44492025-08-26T21:49:02.992ZDEBGnexus (MgsClient): client request background_task = nat_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
44502025-08-26T21:49:02.992ZDEBGnexus (MgsClient): client response background_task = nat_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "48b8b8c7-50d9-4511-9709-8dff75bfbeaa", "content-length": "26", "date": "Tue, 26 Aug 2025 21:49:02 GMT"} })
44512025-08-26T21:49:02.992ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1216 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
44522025-08-26T21:49:02.994ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
44532025-08-26T21:49:03.008ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
44542025-08-26T21:49:03.008ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
44552025-08-26T21:49:03.137ZINFOnexus (BackgroundTasks): VPC route manager sled dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
44562025-08-26T21:49:03.137ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
44572025-08-26T21:49:03.138ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "5f95a300-590b-45b2-8d7d-877448fcba17", "content-length": "372", "date": "Tue, 26 Aug 2025 21:49:03 GMT"} })
44582025-08-26T21:49:03.138ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
44592025-08-26T21:49:03.138ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
44602025-08-26T21:49:03.152ZINFOnexus (BackgroundTasks): instance reincarnation completed successfully background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:113 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0
44612025-08-26T21:49:03.152ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 290.764524ms iteration = 4 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
44622025-08-26T21:49:03.171ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1213 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 zone_address = fd00:1122:3344:101::2
44632025-08-26T21:49:03.171ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
44642025-08-26T21:49:03.171ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "ac8f8627-06d4-420f-937e-5fbd61281aa0", "content-length": "26", "date": "Tue, 26 Aug 2025 21:49:02 GMT"} })
44652025-08-26T21:49:03.171ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1216 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
44662025-08-26T21:49:03.173ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
44672025-08-26T21:49:03.183ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
44682025-08-26T21:49:03.184ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "615eeff3-0d59-46a2-8f63-6c2fb0d58557", "content-length": "691", "date": "Tue, 26 Aug 2025 21:49:03 GMT"} })
44692025-08-26T21:49:03.184ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
44702025-08-26T21:49:03.184ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
44712025-08-26T21:49:03.184ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
44722025-08-26T21:49:03.185ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 324.220739ms iteration = 7 status = {}
44732025-08-26T21:49:03.187ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
44742025-08-26T21:49:03.187ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
44752025-08-26T21:49:03.188ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "2025f2c3-e05c-4735-8ef8-e5678057694e", "date": "Tue, 26 Aug 2025 21:49:03 GMT"} })
44762025-08-26T21:49:03.188ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
44772025-08-26T21:49:03.188ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
44782025-08-26T21:49:03.189ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
44792025-08-26T21:49:03.189ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
44802025-08-26T21:49:03.190ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
44812025-08-26T21:49:03.190ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
44822025-08-26T21:49:03.191ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
44832025-08-26T21:49:03.191ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
44842025-08-26T21:49:03.191ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
44852025-08-26T21:49:03.193ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
44862025-08-26T21:49:03.194ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
44872025-08-26T21:49:03.194ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
44882025-08-26T21:49:03.196ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: dd0c655a-95c2-48fc-b4c3-9733ed4b0faa, lookup_type: ById(dd0c655a-95c2-48fc-b4c3-9733ed4b0faa) } result = Ok(())
44892025-08-26T21:49:03.196ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
44902025-08-26T21:49:03.196ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
44912025-08-26T21:49:03.197ZINFOnexus (BackgroundTasks): VPC route manager sled dd0c655a-95c2-48fc-b4c3-9733ed4b0faa push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
44922025-08-26T21:49:03.197ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
44932025-08-26T21:49:03.197ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "209d31b6-c24d-441d-830f-ebbb9a152f85", "date": "Tue, 26 Aug 2025 21:49:03 GMT"} })
44942025-08-26T21:49:03.198ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 336.457674ms iteration = 7 status = 0
44952025-08-26T21:49:03.198ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 337.189712ms iteration = 7 status = {}
44962025-08-26T21:49:03.223ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 389.569302ms iteration = 7 status = {}
44972025-08-26T21:49:03.288ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
44982025-08-26T21:49:03.288ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "cf7e6d9a-bedf-46fe-8e80-5902ebe28670", "date": "Tue, 26 Aug 2025 21:49:03 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
44992025-08-26T21:49:03.288ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
45002025-08-26T21:49:03.290ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "9883d201-5380-450e-a949-aefab21da94f", "date": "Tue, 26 Aug 2025 21:49:03 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
45012025-08-26T21:49:03.290ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
45022025-08-26T21:49:03.292ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
45032025-08-26T21:49:03.292ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 458.454615ms iteration = 6 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
45042025-08-26T21:49:03.299ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
45052025-08-26T21:49:03.300ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
45062025-08-26T21:49:03.301ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 467.418817ms iteration = 7 status = null
45072025-08-26T21:49:03.303ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
45082025-08-26T21:49:03.303ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "5faff7b8-fafb-4ff3-8a20-83a294d65a8d", "date": "Tue, 26 Aug 2025 21:49:03 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
45092025-08-26T21:49:03.303ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
45102025-08-26T21:49:03.306ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "730c7a36-bb94-4191-9e99-2cb53bf6235e", "date": "Tue, 26 Aug 2025 21:49:03 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
45112025-08-26T21:49:03.306ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
45122025-08-26T21:49:03.308ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "378835b9-de1e-4ef5-9d34-afa94ca4f662", "date": "Tue, 26 Aug 2025 21:49:03 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
45132025-08-26T21:49:03.308ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
45142025-08-26T21:49:03.310ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
45152025-08-26T21:49:03.310ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 476.45568ms iteration = 6 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
45162025-08-26T21:49:03.364ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
45172025-08-26T21:49:03.364ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
45182025-08-26T21:49:03.366ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: d24c5c55-3a21-4a91-aeb4-45362e822df6, rack_id: e37a46f2-f789-4ca4-9841-f8c6dd705964, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(88d84f37-a4c8-4abc-af1f-27987b0d39d9) } rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0
45192025-08-26T21:49:03.366ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
45202025-08-26T21:49:03.366ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
45212025-08-26T21:49:03.379ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1832 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
45222025-08-26T21:49:03.379ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1859 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 88d84f37-a4c8-4abc-af1f-27987b0d39d9
45232025-08-26T21:49:03.380ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
45242025-08-26T21:49:03.380ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
45252025-08-26T21:49:03.380ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
45262025-08-26T21:49:03.380ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
45272025-08-26T21:49:03.380ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
45282025-08-26T21:49:03.380ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2006 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
45292025-08-26T21:49:03.380ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:408 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
45302025-08-26T21:49:03.380ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2042 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
45312025-08-26T21:49:03.380ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
45322025-08-26T21:49:03.380ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
45332025-08-26T21:49:03.381ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
45342025-08-26T21:49:03.384ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0
45352025-08-26T21:49:03.384ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
45362025-08-26T21:49:03.484ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "2d7aa717-8c8d-44d1-bf46-dace01b424c4", "date": "Tue, 26 Aug 2025 21:49:03 GMT"} })
45372025-08-26T21:49:03.485ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch = Switch0
45382025-08-26T21:49:03.485ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
45392025-08-26T21:49:03.485ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "71f870d2-1c75-4736-9c9e-2a150fbfbedd", "content-length": "682", "date": "Tue, 26 Aug 2025 21:49:03 GMT"} })
45402025-08-26T21:49:03.485ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
45412025-08-26T21:49:03.485ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
45422025-08-26T21:49:03.488ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
45432025-08-26T21:49:03.489ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
45442025-08-26T21:49:03.492ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
45452025-08-26T21:49:03.492ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
45462025-08-26T21:49:03.495ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
45472025-08-26T21:49:03.495ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
45482025-08-26T21:49:03.502ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 640.502088ms iteration = 7 status = {}
45492025-08-26T21:49:05.353ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 474 local_addr = [fd00:1122:3344:101::c]:42016 method = GET remote_addr = [fd00:1122:3344:101::d]:37550 req_id = a233fa38-0b79-4bae-9e77-b29fccd5e1fc response_code = 200 uri = /5d0959e5-47a6-4bf3-a52b-57950b9d900d
45502025-08-26T21:49:07.848ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 38 reason = Timeout
45512025-08-26T21:49:07.848ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
45522025-08-26T21:49:07.849ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
45532025-08-26T21:49:07.851ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45542025-08-26T21:49:07.854ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
45552025-08-26T21:49:07.854ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
45562025-08-26T21:49:07.854ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
45572025-08-26T21:49:07.857ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
45582025-08-26T21:49:07.857ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 8.509292ms iteration = 38 status = {"chicken_switches_updated":false}
45592025-08-26T21:49:12.848ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 39 reason = Timeout
45602025-08-26T21:49:12.848ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
45612025-08-26T21:49:12.848ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
45622025-08-26T21:49:12.850ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45632025-08-26T21:49:12.853ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
45642025-08-26T21:49:12.853ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
45652025-08-26T21:49:12.854ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
45662025-08-26T21:49:12.856ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
45672025-08-26T21:49:12.856ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 8.090782ms iteration = 39 status = {"chicken_switches_updated":false}
45682025-08-26T21:49:12.861ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 20 reason = Timeout
45692025-08-26T21:49:12.861ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
45702025-08-26T21:49:12.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
45712025-08-26T21:49:12.873ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 20 reason = Timeout
45722025-08-26T21:49:12.873ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
45732025-08-26T21:49:12.874ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
45742025-08-26T21:49:12.876ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45752025-08-26T21:49:12.878ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
45762025-08-26T21:49:12.878ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
45772025-08-26T21:49:12.878ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
45782025-08-26T21:49:12.881ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
45792025-08-26T21:49:12.881ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
45802025-08-26T21:49:12.883ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45812025-08-26T21:49:12.885ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
45822025-08-26T21:49:12.885ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
45832025-08-26T21:49:12.885ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
45842025-08-26T21:49:12.897ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
45852025-08-26T21:49:12.897ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
45862025-08-26T21:49:12.899ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 37.343932ms iteration = 20 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
45872025-08-26T21:49:12.903ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
45882025-08-26T21:49:12.903ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 29.395533ms iteration = 20 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
45892025-08-26T21:49:15.346ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:1025 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::d]:58938
45902025-08-26T21:49:15.346ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:58938 req_id = fe8b0e54-309e-476c-a74e-3f7e98504fc6 roles = RoleSet { roles: {} } uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?sort_by=id_ascending
45912025-08-26T21:49:15.347ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:58938 req_id = fe8b0e54-309e-476c-a74e-3f7e98504fc6 resource = Database result = Ok(()) uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?sort_by=id_ascending
45922025-08-26T21:49:15.350ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 3181 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:58938 req_id = fe8b0e54-309e-476c-a74e-3f7e98504fc6 response_code = 200 uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?sort_by=id_ascending
45932025-08-26T21:49:15.350ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:58938 req_id = fb5ee826-f774-44aa-b619-bac669e0ec37 roles = RoleSet { roles: {} } uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMzhjYTg4YS1lZGE1LTQwZGEtOGE4MS1hN2UzMzdhMmM4ZmIifX0%3D
45942025-08-26T21:49:15.350ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:58938 req_id = fb5ee826-f774-44aa-b619-bac669e0ec37 resource = Database result = Ok(()) uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMzhjYTg4YS1lZGE1LTQwZGEtOGE4MS1hN2UzMzdhMmM4ZmIifX0%3D
45952025-08-26T21:49:15.352ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 2515 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:58938 req_id = fb5ee826-f774-44aa-b619-bac669e0ec37 response_code = 200 uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMzhjYTg4YS1lZGE1LTQwZGEtOGE4MS1hN2UzMzdhMmM4ZmIifX0%3D
45962025-08-26T21:49:15.353ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 253 local_addr = [fd00:1122:3344:101::c]:42016 method = GET remote_addr = [fd00:1122:3344:101::d]:37550 req_id = 19f3787c-4573-43c1-8920-9c3bd9730a63 response_code = 200 uri = /5d0959e5-47a6-4bf3-a52b-57950b9d900d
45972025-08-26T21:49:17.848ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 40 reason = Timeout
45982025-08-26T21:49:17.848ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
45992025-08-26T21:49:17.848ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
46002025-08-26T21:49:17.850ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
46012025-08-26T21:49:17.853ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
46022025-08-26T21:49:17.853ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
46032025-08-26T21:49:17.853ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
46042025-08-26T21:49:17.855ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
46052025-08-26T21:49:17.855ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.347026ms iteration = 40 status = {"chicken_switches_updated":false}
46062025-08-26T21:49:22.848ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 41 reason = Timeout
46072025-08-26T21:49:22.848ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
46082025-08-26T21:49:22.848ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
46092025-08-26T21:49:22.850ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
46102025-08-26T21:49:22.853ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
46112025-08-26T21:49:22.853ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
46122025-08-26T21:49:22.853ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
46132025-08-26T21:49:22.854ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
46142025-08-26T21:49:22.854ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 6.463466ms iteration = 41 status = {"chicken_switches_updated":false}
46152025-08-26T21:49:22.861ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 21 reason = Timeout
46162025-08-26T21:49:22.861ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
46172025-08-26T21:49:22.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
46182025-08-26T21:49:22.863ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
46192025-08-26T21:49:22.863ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
46202025-08-26T21:49:22.865ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.165024ms iteration = 21 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
46212025-08-26T21:49:22.873ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 21 reason = Timeout
46222025-08-26T21:49:22.873ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
46232025-08-26T21:49:22.874ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
46242025-08-26T21:49:22.875ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
46252025-08-26T21:49:22.878ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
46262025-08-26T21:49:22.878ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
46272025-08-26T21:49:22.878ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
46282025-08-26T21:49:22.881ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
46292025-08-26T21:49:22.881ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
46302025-08-26T21:49:22.882ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
46312025-08-26T21:49:22.885ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
46322025-08-26T21:49:22.885ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
46332025-08-26T21:49:22.885ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
46342025-08-26T21:49:22.900ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
46352025-08-26T21:49:22.900ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 26.312123ms iteration = 21 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
46362025-08-26T21:49:25.353ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 431 local_addr = [fd00:1122:3344:101::c]:42016 method = GET remote_addr = [fd00:1122:3344:101::d]:37550 req_id = e1e36240-28ee-4b64-b798-d845d36b4b78 response_code = 200 uri = /5d0959e5-47a6-4bf3-a52b-57950b9d900d
46372025-08-26T21:49:27.848ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 42 reason = Timeout
46382025-08-26T21:49:27.848ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
46392025-08-26T21:49:27.849ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
46402025-08-26T21:49:27.850ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
46412025-08-26T21:49:27.853ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
46422025-08-26T21:49:27.853ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
46432025-08-26T21:49:27.853ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
46442025-08-26T21:49:27.855ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
46452025-08-26T21:49:27.855ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.214613ms iteration = 42 status = {"chicken_switches_updated":false}
46462025-08-26T21:49:32.834ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 8 reason = Timeout
46472025-08-26T21:49:32.834ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 8 reason = Timeout
46482025-08-26T21:49:32.834ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
46492025-08-26T21:49:32.834ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 8 reason = Timeout
46502025-08-26T21:49:32.834ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
46512025-08-26T21:49:32.835ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
46522025-08-26T21:49:32.835ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
46532025-08-26T21:49:32.837ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1076, tv_nsec: 694782407 } })
46542025-08-26T21:49:32.837ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1076, tv_nsec: 695042583 } })
46552025-08-26T21:49:32.837ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1205
46562025-08-26T21:49:32.837ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1205
46572025-08-26T21:49:32.837ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.153131ms iteration = 8 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
46582025-08-26T21:49:32.847ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 43 reason = Timeout
46592025-08-26T21:49:32.847ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
46602025-08-26T21:49:32.848ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
46612025-08-26T21:49:32.849ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
46622025-08-26T21:49:32.852ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
46632025-08-26T21:49:32.852ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
46642025-08-26T21:49:32.852ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
46652025-08-26T21:49:32.854ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
46662025-08-26T21:49:32.854ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 6.959097ms iteration = 43 status = {"chicken_switches_updated":false}
46672025-08-26T21:49:32.860ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 8 reason = Timeout
46682025-08-26T21:49:32.860ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
46692025-08-26T21:49:32.860ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
46702025-08-26T21:49:32.860ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 8 reason = Timeout
46712025-08-26T21:49:32.860ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 8 reason = Timeout
46722025-08-26T21:49:32.860ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
46732025-08-26T21:49:32.860ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
46742025-08-26T21:49:32.860ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 8 reason = Timeout
46752025-08-26T21:49:32.860ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
46762025-08-26T21:49:32.860ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 9 reason = Timeout
46772025-08-26T21:49:32.860ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
46782025-08-26T21:49:32.860ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 8 reason = Timeout
46792025-08-26T21:49:32.860ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
46802025-08-26T21:49:32.860ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
46812025-08-26T21:49:32.860ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 8 reason = Timeout
46822025-08-26T21:49:32.860ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 8 reason = Timeout
46832025-08-26T21:49:32.860ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
46842025-08-26T21:49:32.860ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 22 reason = Timeout
46852025-08-26T21:49:32.860ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
46862025-08-26T21:49:32.860ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
46872025-08-26T21:49:32.860ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 8 reason = Timeout
46882025-08-26T21:49:32.860ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
46892025-08-26T21:49:32.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
46902025-08-26T21:49:32.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
46912025-08-26T21:49:32.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
46922025-08-26T21:49:32.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
46932025-08-26T21:49:32.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
46942025-08-26T21:49:32.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
46952025-08-26T21:49:32.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
46962025-08-26T21:49:32.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
46972025-08-26T21:49:32.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
46982025-08-26T21:49:32.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
46992025-08-26T21:49:32.862ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 8 reason = Timeout
47002025-08-26T21:49:32.862ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 8 reason = Timeout
47012025-08-26T21:49:32.862ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 8 reason = Timeout
47022025-08-26T21:49:32.862ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
47032025-08-26T21:49:32.862ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 11.15\u{b5}s iteration = 8 status = {"disabled":true,"errors":[],"sps":[]}
47042025-08-26T21:49:32.862ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 8 reason = Timeout
47052025-08-26T21:49:32.862ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
47062025-08-26T21:49:32.862ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
47072025-08-26T21:49:32.862ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47082025-08-26T21:49:32.862ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 8 reason = Timeout
47092025-08-26T21:49:32.862ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
47102025-08-26T21:49:32.862ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 8 reason = Timeout
47112025-08-26T21:49:32.862ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
47122025-08-26T21:49:32.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
47132025-08-26T21:49:32.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
47142025-08-26T21:49:32.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
47152025-08-26T21:49:32.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
47162025-08-26T21:49:32.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
47172025-08-26T21:49:32.862ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47182025-08-26T21:49:32.863ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47192025-08-26T21:49:32.863ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47202025-08-26T21:49:32.863ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47212025-08-26T21:49:32.863ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47222025-08-26T21:49:32.864ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 2.110147ms iteration = 8 status = {"errors":[],"garbage_collect_requested":[]}
47232025-08-26T21:49:32.864ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
47242025-08-26T21:49:32.864ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
47252025-08-26T21:49:32.866ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
47262025-08-26T21:49:32.866ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
47272025-08-26T21:49:32.866ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
47282025-08-26T21:49:32.866ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
47292025-08-26T21:49:32.866ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
47302025-08-26T21:49:32.866ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
47312025-08-26T21:49:32.867ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
47322025-08-26T21:49:32.867ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
47332025-08-26T21:49:32.867ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
47342025-08-26T21:49:32.867ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
47352025-08-26T21:49:32.867ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
47362025-08-26T21:49:32.867ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
47372025-08-26T21:49:32.867ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
47382025-08-26T21:49:32.867ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
47392025-08-26T21:49:32.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
47402025-08-26T21:49:32.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
47412025-08-26T21:49:32.869ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
47422025-08-26T21:49:32.869ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
47432025-08-26T21:49:32.869ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
47442025-08-26T21:49:32.869ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
47452025-08-26T21:49:32.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
47462025-08-26T21:49:32.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
47472025-08-26T21:49:32.872ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 11.691564ms iteration = 8 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
47482025-08-26T21:49:32.872ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1076, tv_nsec: 730274037 } })
47492025-08-26T21:49:32.872ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 22 reason = Timeout
47502025-08-26T21:49:32.872ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
47512025-08-26T21:49:32.872ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
47522025-08-26T21:49:32.873ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
47532025-08-26T21:49:32.873ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1205 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
47542025-08-26T21:49:32.873ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
47552025-08-26T21:49:32.873ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
47562025-08-26T21:49:32.873ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
47572025-08-26T21:49:32.878ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
47582025-08-26T21:49:32.878ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
47592025-08-26T21:49:32.909ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 47.167793ms iteration = 8 status = {"errors":[],"finish_invoked_ok":[]}
47602025-08-26T21:49:32.919ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
47612025-08-26T21:49:32.919ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
47622025-08-26T21:49:32.919ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
47632025-08-26T21:49:32.919ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
47642025-08-26T21:49:32.920ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
47652025-08-26T21:49:32.921ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
47662025-08-26T21:49:32.921ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 59.00588ms iteration = 8 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
47672025-08-26T21:49:32.922ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47682025-08-26T21:49:32.922ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
47692025-08-26T21:49:32.922ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
47702025-08-26T21:49:32.922ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
47712025-08-26T21:49:32.923ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
47722025-08-26T21:49:32.924ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:550 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
47732025-08-26T21:49:32.924ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 63.777128ms iteration = 8 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
47742025-08-26T21:49:32.925ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
47752025-08-26T21:49:32.925ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
47762025-08-26T21:49:32.925ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
47772025-08-26T21:49:32.928ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47782025-08-26T21:49:32.929ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 67.298827ms iteration = 8 status = {"errors":[],"requests_created_ok":[]}
47792025-08-26T21:49:32.930ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
47802025-08-26T21:49:32.930ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
47812025-08-26T21:49:32.930ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1213 zone_address = fd00:1122:3344:101::2
47822025-08-26T21:49:32.930ZDEBGnexus (MgsClient): client request background_task = nat_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
47832025-08-26T21:49:32.930ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
47842025-08-26T21:49:32.931ZDEBGnexus (MgsClient): client response background_task = nat_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "3fd28208-5897-4cc2-ba50-dec52c7f04a1", "content-length": "26", "date": "Tue, 26 Aug 2025 21:49:32 GMT"} })
47852025-08-26T21:49:32.931ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47862025-08-26T21:49:32.931ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1216 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
47872025-08-26T21:49:32.932ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1213 zone_address = fd00:1122:3344:101::2
47882025-08-26T21:49:32.932ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
47892025-08-26T21:49:32.933ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "5eb875d4-f9ae-4dde-931e-4505eb6cf9ca", "content-length": "26", "date": "Tue, 26 Aug 2025 21:49:32 GMT"} })
47902025-08-26T21:49:32.933ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1216 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
47912025-08-26T21:49:32.933ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
47922025-08-26T21:49:32.934ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
47932025-08-26T21:49:32.934ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
47942025-08-26T21:49:32.934ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
47952025-08-26T21:49:32.934ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
47962025-08-26T21:49:32.937ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
47972025-08-26T21:49:32.937ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47982025-08-26T21:49:32.937ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
47992025-08-26T21:49:32.938ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
48002025-08-26T21:49:32.939ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
48012025-08-26T21:49:32.939ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
48022025-08-26T21:49:32.939ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
48032025-08-26T21:49:32.939ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
48042025-08-26T21:49:32.940ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
48052025-08-26T21:49:32.940ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
48062025-08-26T21:49:32.960ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1213 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 zone_address = fd00:1122:3344:101::2
48072025-08-26T21:49:32.960ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
48082025-08-26T21:49:32.960ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "063efac8-a22f-4625-9345-2ee1fe3956ac", "content-length": "26", "date": "Tue, 26 Aug 2025 21:49:32 GMT"} })
48092025-08-26T21:49:32.960ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1216 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
48102025-08-26T21:49:32.962ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
48112025-08-26T21:49:32.971ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
48122025-08-26T21:49:32.971ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
48132025-08-26T21:49:32.971ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 110.632794ms iteration = 8 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
48142025-08-26T21:49:32.972ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:407
48152025-08-26T21:49:32.972ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 111.682547ms iteration = 8 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
48162025-08-26T21:49:32.981ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 118.991992ms iteration = 8 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
48172025-08-26T21:49:32.983ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
48182025-08-26T21:49:32.984ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 123.480363ms iteration = 22 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
48192025-08-26T21:49:32.986ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
48202025-08-26T21:49:32.986ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
48212025-08-26T21:49:32.986ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 125.202582ms iteration = 9 status = {"physical_disks_added":0}
48222025-08-26T21:49:32.986ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
48232025-08-26T21:49:32.999ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
48242025-08-26T21:49:33.000ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
48252025-08-26T21:49:33.001ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
48262025-08-26T21:49:33.003ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
48272025-08-26T21:49:33.003ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 130.594714ms iteration = 22 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
48282025-08-26T21:49:33.003ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
48292025-08-26T21:49:33.003ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
48302025-08-26T21:49:33.003ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
48312025-08-26T21:49:33.005ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
48322025-08-26T21:49:33.007ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
48332025-08-26T21:49:33.007ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
48342025-08-26T21:49:33.007ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
48352025-08-26T21:49:33.031ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
48362025-08-26T21:49:33.031ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
48372025-08-26T21:49:33.034ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 199.431296ms iteration = 8 status = null
48382025-08-26T21:49:33.035ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 201.113253ms iteration = 8 status = {}
48392025-08-26T21:49:33.079ZINFOnexus (BackgroundTasks): VPC route manager sled dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
48402025-08-26T21:49:33.079ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
48412025-08-26T21:49:33.080ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "4aa93bc6-f515-412c-99ae-d910c4ede316", "content-length": "372", "date": "Tue, 26 Aug 2025 21:49:33 GMT"} })
48422025-08-26T21:49:33.080ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
48432025-08-26T21:49:33.080ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
48442025-08-26T21:49:33.130ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
48452025-08-26T21:49:33.130ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
48462025-08-26T21:49:33.130ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "b2025179-c173-4aad-bae8-1cb16b62ad3e", "date": "Tue, 26 Aug 2025 21:49:33 GMT"} })
48472025-08-26T21:49:33.130ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
48482025-08-26T21:49:33.130ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
48492025-08-26T21:49:33.133ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
48502025-08-26T21:49:33.133ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
48512025-08-26T21:49:33.136ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
48522025-08-26T21:49:33.136ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
48532025-08-26T21:49:33.140ZINFOnexus (BackgroundTasks): VPC route manager sled dd0c655a-95c2-48fc-b4c3-9733ed4b0faa push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
48542025-08-26T21:49:33.140ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
48552025-08-26T21:49:33.140ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "085575c5-b501-4207-ac3a-516261feb98e", "date": "Tue, 26 Aug 2025 21:49:33 GMT"} })
48562025-08-26T21:49:33.141ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
48572025-08-26T21:49:33.141ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 281.099416ms iteration = 8 status = {}
48582025-08-26T21:49:33.142ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "64bf6bf2-3976-48d9-b1cb-a06637414049", "content-length": "691", "date": "Tue, 26 Aug 2025 21:49:33 GMT"} })
48592025-08-26T21:49:33.142ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
48602025-08-26T21:49:33.142ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
48612025-08-26T21:49:33.142ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
48622025-08-26T21:49:33.143ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 282.672942ms iteration = 8 status = {}
48632025-08-26T21:49:33.158ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
48642025-08-26T21:49:33.158ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
48652025-08-26T21:49:33.160ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: d24c5c55-3a21-4a91-aeb4-45362e822df6, rack_id: e37a46f2-f789-4ca4-9841-f8c6dd705964, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(88d84f37-a4c8-4abc-af1f-27987b0d39d9) } rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0
48662025-08-26T21:49:33.160ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
48672025-08-26T21:49:33.160ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
48682025-08-26T21:49:33.171ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1832 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
48692025-08-26T21:49:33.171ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1859 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 88d84f37-a4c8-4abc-af1f-27987b0d39d9
48702025-08-26T21:49:33.172ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
48712025-08-26T21:49:33.172ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
48722025-08-26T21:49:33.172ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
48732025-08-26T21:49:33.172ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
48742025-08-26T21:49:33.172ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
48752025-08-26T21:49:33.172ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2006 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
48762025-08-26T21:49:33.172ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:408 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
48772025-08-26T21:49:33.172ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2042 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
48782025-08-26T21:49:33.172ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
48792025-08-26T21:49:33.172ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
48802025-08-26T21:49:33.172ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
48812025-08-26T21:49:33.175ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0
48822025-08-26T21:49:33.175ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
48832025-08-26T21:49:33.204ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
48842025-08-26T21:49:33.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
48852025-08-26T21:49:33.206ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
48862025-08-26T21:49:33.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
48872025-08-26T21:49:33.208ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
48882025-08-26T21:49:33.211ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: dd0c655a-95c2-48fc-b4c3-9733ed4b0faa, lookup_type: ById(dd0c655a-95c2-48fc-b4c3-9733ed4b0faa) } result = Ok(())
48892025-08-26T21:49:33.212ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
48902025-08-26T21:49:33.212ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
48912025-08-26T21:49:33.214ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 353.228023ms iteration = 8 status = 0
48922025-08-26T21:49:33.284ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "79987dbc-766c-4ac5-be5c-d5d6e1b51950", "date": "Tue, 26 Aug 2025 21:49:33 GMT"} })
48932025-08-26T21:49:33.284ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch = Switch0
48942025-08-26T21:49:33.284ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
48952025-08-26T21:49:33.285ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "c47ce980-815a-4391-9a6b-95ea65c57fb0", "content-length": "682", "date": "Tue, 26 Aug 2025 21:49:33 GMT"} })
48962025-08-26T21:49:33.285ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
48972025-08-26T21:49:33.285ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
48982025-08-26T21:49:33.288ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
48992025-08-26T21:49:33.289ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
49002025-08-26T21:49:33.292ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
49012025-08-26T21:49:33.292ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
49022025-08-26T21:49:33.294ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
49032025-08-26T21:49:33.294ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
49042025-08-26T21:49:33.301ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 440.772385ms iteration = 8 status = {}
49052025-08-26T21:49:33.334ZDEBGnexus (producer-registration-task): registering as oximeter producer with Nexus registration_count = 1
49062025-08-26T21:49:33.334ZDEBGnexus (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::c]:12221
49072025-08-26T21:49:33.364ZDEBGnexus (producer-registration-task): client request body = Some(Body) method = POST uri = http://[fd00:1122:3344:101::c]:12221/metrics/producers
49082025-08-26T21:49:33.364ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:1025 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::c]:54314
49092025-08-26T21:49:33.364ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:54314 req_id = 157b8251-8e8f-41fc-9044-85185c7b1137 roles = RoleSet { roles: {} } uri = /metrics/producers
49102025-08-26T21:49:33.364ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:54314 req_id = 157b8251-8e8f-41fc-9044-85185c7b1137 resource = Database result = Ok(()) uri = /metrics/producers
49112025-08-26T21:49:33.418ZINFOnexus: assigned collector to new producer collector_id = f843ff0e-eef8-49c4-b428-23e15df1def3 file = nexus/src/app/oximeter.rs:76 producer_id = 5d0959e5-47a6-4bf3-a52b-57950b9d900d
49122025-08-26T21:49:33.418ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 54394 local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:54314 req_id = 157b8251-8e8f-41fc-9044-85185c7b1137 response_code = 201 uri = /metrics/producers
49132025-08-26T21:49:33.419ZDEBGnexus (producer-registration-task): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::c]:12221/metrics/producers", status: 201, headers: {"content-type": "application/json", "x-request-id": "157b8251-8e8f-41fc-9044-85185c7b1137", "content-length": "41", "date": "Tue, 26 Aug 2025 21:49:33 GMT"} })
49142025-08-26T21:49:33.420ZDEBGnexus (producer-registration-task): successfully registered with Nexus, pausing until time to renew lease lease_duration = 600s wait_period = 150s
49152025-08-26T21:49:35.353ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 311 local_addr = [fd00:1122:3344:101::c]:42016 method = GET remote_addr = [fd00:1122:3344:101::d]:37550 req_id = 19dfa4a4-b054-4a94-be46-83fe102ec7f7 response_code = 200 uri = /5d0959e5-47a6-4bf3-a52b-57950b9d900d
49162025-08-26T21:49:37.848ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 44 reason = Timeout
49172025-08-26T21:49:37.848ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
49182025-08-26T21:49:37.848ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
49192025-08-26T21:49:37.850ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
49202025-08-26T21:49:37.853ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
49212025-08-26T21:49:37.853ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
49222025-08-26T21:49:37.853ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
49232025-08-26T21:49:37.856ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
49242025-08-26T21:49:37.856ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.899378ms iteration = 44 status = {"chicken_switches_updated":false}
49252025-08-26T21:49:42.848ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 45 reason = Timeout
49262025-08-26T21:49:42.848ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
49272025-08-26T21:49:42.848ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
49282025-08-26T21:49:42.850ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
49292025-08-26T21:49:42.852ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
49302025-08-26T21:49:42.852ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
49312025-08-26T21:49:42.853ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
49322025-08-26T21:49:42.854ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
49332025-08-26T21:49:42.854ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 6.494467ms iteration = 45 status = {"chicken_switches_updated":false}
49342025-08-26T21:49:42.860ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 23 reason = Timeout
49352025-08-26T21:49:42.860ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
49362025-08-26T21:49:42.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
49372025-08-26T21:49:42.863ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
49382025-08-26T21:49:42.863ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
49392025-08-26T21:49:42.865ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.509781ms iteration = 23 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
49402025-08-26T21:49:42.873ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 23 reason = Timeout
49412025-08-26T21:49:42.873ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
49422025-08-26T21:49:42.873ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
49432025-08-26T21:49:42.875ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
49442025-08-26T21:49:42.878ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
49452025-08-26T21:49:42.878ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
49462025-08-26T21:49:42.878ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
49472025-08-26T21:49:42.881ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
49482025-08-26T21:49:42.881ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
49492025-08-26T21:49:42.883ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
49502025-08-26T21:49:42.885ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
49512025-08-26T21:49:42.885ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
49522025-08-26T21:49:42.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
49532025-08-26T21:49:42.899ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
49542025-08-26T21:49:42.899ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 26.289853ms iteration = 23 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
49552025-08-26T21:49:45.346ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:58938 req_id = 4fa82a95-2b2e-4365-be58-017b4f49c92f roles = RoleSet { roles: {} } uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?sort_by=id_ascending
49562025-08-26T21:49:45.346ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:58938 req_id = 4fa82a95-2b2e-4365-be58-017b4f49c92f resource = Database result = Ok(()) uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?sort_by=id_ascending
49572025-08-26T21:49:45.348ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 2693 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:58938 req_id = 4fa82a95-2b2e-4365-be58-017b4f49c92f response_code = 200 uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?sort_by=id_ascending
49582025-08-26T21:49:45.349ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:58938 req_id = 929f4dec-ba2f-45dd-8e23-8ffb1b40b5c8 roles = RoleSet { roles: {} } uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMzhjYTg4YS1lZGE1LTQwZGEtOGE4MS1hN2UzMzdhMmM4ZmIifX0%3D
49592025-08-26T21:49:45.349ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:58938 req_id = 929f4dec-ba2f-45dd-8e23-8ffb1b40b5c8 resource = Database result = Ok(()) uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMzhjYTg4YS1lZGE1LTQwZGEtOGE4MS1hN2UzMzdhMmM4ZmIifX0%3D
49602025-08-26T21:49:45.351ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 2373 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:58938 req_id = 929f4dec-ba2f-45dd-8e23-8ffb1b40b5c8 response_code = 200 uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMzhjYTg4YS1lZGE1LTQwZGEtOGE4MS1hN2UzMzdhMmM4ZmIifX0%3D
49612025-08-26T21:49:45.353ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 295 local_addr = [fd00:1122:3344:101::c]:42016 method = GET remote_addr = [fd00:1122:3344:101::d]:37550 req_id = 9585b7f2-e5bc-40da-9e19-3b010bcb5854 response_code = 200 uri = /5d0959e5-47a6-4bf3-a52b-57950b9d900d
49622025-08-26T21:49:47.848ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 46 reason = Timeout
49632025-08-26T21:49:47.848ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
49642025-08-26T21:49:47.848ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
49652025-08-26T21:49:47.850ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
49662025-08-26T21:49:47.853ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
49672025-08-26T21:49:47.853ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
49682025-08-26T21:49:47.853ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
49692025-08-26T21:49:47.854ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
49702025-08-26T21:49:47.854ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 6.591308ms iteration = 46 status = {"chicken_switches_updated":false}
49712025-08-26T21:49:52.847ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 47 reason = Timeout
49722025-08-26T21:49:52.847ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
49732025-08-26T21:49:52.848ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
49742025-08-26T21:49:52.861ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 24 reason = Timeout
49752025-08-26T21:49:52.861ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
49762025-08-26T21:49:52.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
49772025-08-26T21:49:52.863ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
49782025-08-26T21:49:52.864ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
49792025-08-26T21:49:52.865ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.555882ms iteration = 24 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
49802025-08-26T21:49:52.872ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 24 reason = Timeout
49812025-08-26T21:49:52.872ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
49822025-08-26T21:49:52.873ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
49832025-08-26T21:49:52.874ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
49842025-08-26T21:49:52.879ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
49852025-08-26T21:49:52.879ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
49862025-08-26T21:49:52.879ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
49872025-08-26T21:49:52.880ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
49882025-08-26T21:49:52.881ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
49892025-08-26T21:49:52.883ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
49902025-08-26T21:49:52.885ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
49912025-08-26T21:49:52.885ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
49922025-08-26T21:49:52.885ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
49932025-08-26T21:49:52.891ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
49942025-08-26T21:49:52.894ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
49952025-08-26T21:49:52.894ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
49962025-08-26T21:49:52.894ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
49972025-08-26T21:49:52.896ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
49982025-08-26T21:49:52.896ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 48.411021ms iteration = 47 status = {"chicken_switches_updated":false}
49992025-08-26T21:49:52.902ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
50002025-08-26T21:49:52.902ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 29.654168ms iteration = 24 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
50012025-08-26T21:49:55.353ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 461 local_addr = [fd00:1122:3344:101::c]:42016 method = GET remote_addr = [fd00:1122:3344:101::d]:37550 req_id = d8942b35-a882-48dc-aa52-3a3b25ae49fa response_code = 200 uri = /5d0959e5-47a6-4bf3-a52b-57950b9d900d
50022025-08-26T21:49:57.847ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 48 reason = Timeout
50032025-08-26T21:49:57.847ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
50042025-08-26T21:49:57.848ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
50052025-08-26T21:49:57.849ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
50062025-08-26T21:49:57.852ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
50072025-08-26T21:49:57.852ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
50082025-08-26T21:49:57.852ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
50092025-08-26T21:49:57.854ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
50102025-08-26T21:49:57.854ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 6.227451ms iteration = 48 status = {"chicken_switches_updated":false}
50112025-08-26T21:50:02.834ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 5 reason = Timeout
50122025-08-26T21:50:02.834ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 5 reason = Timeout
50132025-08-26T21:50:02.834ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 5 reason = Timeout
50142025-08-26T21:50:02.834ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
50152025-08-26T21:50:02.834ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-08-26 21:40:02.834274897 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
50162025-08-26T21:50:02.834ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
50172025-08-26T21:50:02.834ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 5 reason = Timeout
50182025-08-26T21:50:02.834ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 5 reason = Timeout
50192025-08-26T21:50:02.834ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
50202025-08-26T21:50:02.834ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
50212025-08-26T21:50:02.834ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 7 reason = Timeout
50222025-08-26T21:50:02.834ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 5 reason = Timeout
50232025-08-26T21:50:02.834ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
50242025-08-26T21:50:02.834ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 9 reason = Timeout
50252025-08-26T21:50:02.834ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
50262025-08-26T21:50:02.834ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
50272025-08-26T21:50:02.834ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 9 reason = Timeout
50282025-08-26T21:50:02.834ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 9 reason = Timeout
50292025-08-26T21:50:02.834ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
50302025-08-26T21:50:02.834ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 7 reason = Timeout
50312025-08-26T21:50:02.834ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
50322025-08-26T21:50:02.834ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
50332025-08-26T21:50:02.834ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
50342025-08-26T21:50:02.834ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
50352025-08-26T21:50:02.834ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
50362025-08-26T21:50:02.834ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("852fd1a3-de9a-47e8-a667-5e8dab880ad3.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("c68f1501-294c-4874-92ca-8fc0546a5870.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("d741a72f-8745-4723-9e80-e1e72987266b.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("d741a72f-8745-4723-9e80-e1e72987266b.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:3::1)) }], valid_until: Instant { tv_sec: 1106, tv_nsec: 692489085 } })
50372025-08-26T21:50:02.834ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
50382025-08-26T21:50:02.834ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("60abf731-31e1-4111-a3e2-65da89eaed87.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("84c849eb-0949-41d9-ad02-9a5509144564.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("84c849eb-0949-41d9-ad02-9a5509144564.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::8)) }], valid_until: Instant { tv_sec: 1106, tv_nsec: 692604818 } })
50392025-08-26T21:50:02.835ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
50402025-08-26T21:50:02.835ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.100955ms iteration = 5 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
50412025-08-26T21:50:02.835ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } dns_group = internal
50422025-08-26T21:50:02.835ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.231548ms iteration = 5 status = {"addresses":["[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:3::1]:5353"]}
50432025-08-26T21:50:02.835ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
50442025-08-26T21:50:02.835ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
50452025-08-26T21:50:02.835ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
50462025-08-26T21:50:02.836ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
50472025-08-26T21:50:02.836ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 6 reason = Timeout
50482025-08-26T21:50:02.836ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:91 reason = blueprint disabled target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8
50492025-08-26T21:50:02.837ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 12.63\u{b5}s iteration = 6 status = {"enabled":false,"target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8"}
50502025-08-26T21:50:02.837ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-08-26 21:40:02.834274897 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
50512025-08-26T21:50:02.837ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
50522025-08-26T21:50:02.837ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 2.729351ms iteration = 5 status = {"expiration":"2025-08-26T21:40:02.834274897Z","pruned":[]}
50532025-08-26T21:50:02.837ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1106, tv_nsec: 695067713 } })
50542025-08-26T21:50:02.837ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1205
50552025-08-26T21:50:02.837ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
50562025-08-26T21:50:02.837ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1106, tv_nsec: 695581295 } })
50572025-08-26T21:50:02.838ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1205
50582025-08-26T21:50:02.838ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 4.162114ms iteration = 9 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
50592025-08-26T21:50:02.839ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
50602025-08-26T21:50:02.839ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
50612025-08-26T21:50:02.839ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
50622025-08-26T21:50:02.841ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
50632025-08-26T21:50:02.841ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
50642025-08-26T21:50:02.841ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
50652025-08-26T21:50:02.843ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
50662025-08-26T21:50:02.846ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
50672025-08-26T21:50:02.846ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
50682025-08-26T21:50:02.847ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
50692025-08-26T21:50:02.848ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 49 reason = Timeout
50702025-08-26T21:50:02.848ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
50712025-08-26T21:50:02.848ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
50722025-08-26T21:50:02.850ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
50732025-08-26T21:50:02.853ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
50742025-08-26T21:50:02.853ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
50752025-08-26T21:50:02.853ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
50762025-08-26T21:50:02.855ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
50772025-08-26T21:50:02.855ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.060649ms iteration = 49 status = {"chicken_switches_updated":false}
50782025-08-26T21:50:02.860ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 7 reason = Timeout
50792025-08-26T21:50:02.860ZDEBGnexus (BackgroundTasks): blueprint planning disabled, doing nothing background_task = blueprint_planner
50802025-08-26T21:50:02.860ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 10.29\u{b5}s iteration = 7 status = "Disabled"
50812025-08-26T21:50:02.860ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 9 reason = Timeout
50822025-08-26T21:50:02.860ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 10 reason = Timeout
50832025-08-26T21:50:02.861ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
50842025-08-26T21:50:02.861ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
50852025-08-26T21:50:02.861ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 5 reason = Timeout
50862025-08-26T21:50:02.861ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
50872025-08-26T21:50:02.861ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 5 reason = Timeout
50882025-08-26T21:50:02.861ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
50892025-08-26T21:50:02.861ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 5 reason = Timeout
50902025-08-26T21:50:02.861ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:53
50912025-08-26T21:50:02.861ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 9 reason = Timeout
50922025-08-26T21:50:02.861ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
50932025-08-26T21:50:02.861ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
50942025-08-26T21:50:02.861ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 9 reason = Timeout
50952025-08-26T21:50:02.861ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
50962025-08-26T21:50:02.861ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 9 reason = Timeout
50972025-08-26T21:50:02.861ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
50982025-08-26T21:50:02.861ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
50992025-08-26T21:50:02.861ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 9 reason = Timeout
51002025-08-26T21:50:02.861ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
51012025-08-26T21:50:02.861ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 9 reason = Timeout
51022025-08-26T21:50:02.861ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
51032025-08-26T21:50:02.861ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 9 reason = Timeout
51042025-08-26T21:50:02.861ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
51052025-08-26T21:50:02.861ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
51062025-08-26T21:50:02.861ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 25 reason = Timeout
51072025-08-26T21:50:02.861ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
51082025-08-26T21:50:02.861ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 9 reason = Timeout
51092025-08-26T21:50:02.861ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
51102025-08-26T21:50:02.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
51112025-08-26T21:50:02.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
51122025-08-26T21:50:02.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
51132025-08-26T21:50:02.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
51142025-08-26T21:50:02.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
51152025-08-26T21:50:02.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
51162025-08-26T21:50:02.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
51172025-08-26T21:50:02.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
51182025-08-26T21:50:02.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
51192025-08-26T21:50:02.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
51202025-08-26T21:50:02.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
51212025-08-26T21:50:02.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
51222025-08-26T21:50:02.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
51232025-08-26T21:50:02.862ZDEBGnexus (BackgroundTasks): activating background_task = webhook_deliverator iteration = 5 reason = Timeout
51242025-08-26T21:50:02.862ZDEBGnexus (BackgroundTasks): roles background_task = webhook_deliverator roles = RoleSet { roles: {} }
51252025-08-26T21:50:02.862ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 9 reason = Timeout
51262025-08-26T21:50:02.862ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
51272025-08-26T21:50:02.862ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 9 reason = Timeout
51282025-08-26T21:50:02.862ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 9 reason = Timeout
51292025-08-26T21:50:02.862ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
51302025-08-26T21:50:02.862ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
51312025-08-26T21:50:02.862ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 9 reason = Timeout
51322025-08-26T21:50:02.862ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
51332025-08-26T21:50:02.862ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 9 reason = Timeout
51342025-08-26T21:50:02.862ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
51352025-08-26T21:50:02.862ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 5 reason = Timeout
51362025-08-26T21:50:02.862ZDEBGnexus (BackgroundTasks): activating background_task = alert_dispatcher iteration = 5 reason = Timeout
51372025-08-26T21:50:02.862ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
51382025-08-26T21:50:02.862ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
51392025-08-26T21:50:02.862ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 9 reason = Timeout
51402025-08-26T21:50:02.862ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 10.691\u{b5}s iteration = 9 status = {"disabled":true,"errors":[],"sps":[]}
51412025-08-26T21:50:02.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = webhook_deliverator resource = Database result = Ok(())
51422025-08-26T21:50:02.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
51432025-08-26T21:50:02.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
51442025-08-26T21:50:02.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
51452025-08-26T21:50:02.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
51462025-08-26T21:50:02.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = alert_dispatcher resource = Database result = Ok(())
51472025-08-26T21:50:02.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
51482025-08-26T21:50:02.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
51492025-08-26T21:50:02.863ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
51502025-08-26T21:50:02.863ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
51512025-08-26T21:50:02.863ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
51522025-08-26T21:50:02.864ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
51532025-08-26T21:50:02.865ZINFOnexus (BackgroundTasks): all webhook delivery tasks completed background_task = webhook_deliverator file = nexus/src/app/background/tasks/webhook_deliverator.rs:204 num_receivers = 0
51542025-08-26T21:50:02.865ZDEBGnexus (BackgroundTasks): activation complete background_task = webhook_deliverator elapsed = 2.826323ms iteration = 5 status = {"by_rx":{},"error":null}
51552025-08-26T21:50:02.865ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 2.852124ms iteration = 5 status = {"errors":[],"found_port_ok":[]}
51562025-08-26T21:50:02.867ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 4.861159ms iteration = 9 status = {"errors":[],"garbage_collect_requested":[]}
51572025-08-26T21:50:02.867ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
51582025-08-26T21:50:02.867ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
51592025-08-26T21:50:02.867ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
51602025-08-26T21:50:02.867ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
51612025-08-26T21:50:02.867ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
51622025-08-26T21:50:02.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
51632025-08-26T21:50:02.868ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
51642025-08-26T21:50:02.868ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
51652025-08-26T21:50:02.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
51662025-08-26T21:50:02.868ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
51672025-08-26T21:50:02.868ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
51682025-08-26T21:50:02.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
51692025-08-26T21:50:02.869ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
51702025-08-26T21:50:02.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = alert_dispatcher resource = Database result = Ok(())
51712025-08-26T21:50:02.872ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
51722025-08-26T21:50:02.873ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
51732025-08-26T21:50:02.873ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 25 reason = Timeout
51742025-08-26T21:50:02.873ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
51752025-08-26T21:50:02.873ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
51762025-08-26T21:50:02.874ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
51772025-08-26T21:50:02.874ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
51782025-08-26T21:50:02.874ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
51792025-08-26T21:50:02.877ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
51802025-08-26T21:50:02.882ZDEBGnexus (BackgroundTasks): activation complete background_task = alert_dispatcher elapsed = 19.52169ms iteration = 5 status = {"dispatched":[],"errors":[],"glob_version":"183.0.0","globs_reprocessed":{},"no_receivers":[]}
51812025-08-26T21:50:02.885ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
51822025-08-26T21:50:02.887ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
51832025-08-26T21:50:02.888ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
51842025-08-26T21:50:02.888ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
51852025-08-26T21:50:02.888ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
51862025-08-26T21:50:02.891ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
51872025-08-26T21:50:02.891ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
51882025-08-26T21:50:02.892ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
51892025-08-26T21:50:02.916ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
51902025-08-26T21:50:02.916ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
51912025-08-26T21:50:02.916ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
51922025-08-26T21:50:02.916ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
51932025-08-26T21:50:02.916ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
51942025-08-26T21:50:02.916ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
51952025-08-26T21:50:02.917ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
51962025-08-26T21:50:02.918ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
51972025-08-26T21:50:02.919ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
51982025-08-26T21:50:02.921ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
51992025-08-26T21:50:02.921ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
52002025-08-26T21:50:02.922ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
52012025-08-26T21:50:02.923ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
52022025-08-26T21:50:02.923ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
52032025-08-26T21:50:02.923ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
52042025-08-26T21:50:02.923ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
52052025-08-26T21:50:02.924ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
52062025-08-26T21:50:02.924ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
52072025-08-26T21:50:02.924ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 62.453927ms iteration = 9 status = {"errors":[],"finish_invoked_ok":[]}
52082025-08-26T21:50:02.925ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
52092025-08-26T21:50:02.925ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
52102025-08-26T21:50:02.925ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
52112025-08-26T21:50:02.925ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
52122025-08-26T21:50:02.928ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
52132025-08-26T21:50:02.928ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
52142025-08-26T21:50:02.928ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
52152025-08-26T21:50:02.928ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
52162025-08-26T21:50:02.928ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
52172025-08-26T21:50:02.928ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
52182025-08-26T21:50:02.928ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
52192025-08-26T21:50:02.929ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
52202025-08-26T21:50:02.929ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
52212025-08-26T21:50:02.929ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
52222025-08-26T21:50:02.929ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
52232025-08-26T21:50:02.932ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
52242025-08-26T21:50:02.932ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
52252025-08-26T21:50:02.933ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
52262025-08-26T21:50:02.934ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1106, tv_nsec: 792505190 } })
52272025-08-26T21:50:02.935ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1205 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
52282025-08-26T21:50:02.935ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
52292025-08-26T21:50:02.935ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
52302025-08-26T21:50:02.935ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
52312025-08-26T21:50:02.940ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
52322025-08-26T21:50:02.940ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:170 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
52332025-08-26T21:50:02.940ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 79.87285ms iteration = 5 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
52342025-08-26T21:50:02.945ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
52352025-08-26T21:50:02.945ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
52362025-08-26T21:50:02.945ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
52372025-08-26T21:50:02.945ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:550 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
52382025-08-26T21:50:02.945ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 84.859933ms iteration = 9 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
52392025-08-26T21:50:02.948ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
52402025-08-26T21:50:02.949ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
52412025-08-26T21:50:02.949ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
52422025-08-26T21:50:02.949ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
52432025-08-26T21:50:02.949ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
52442025-08-26T21:50:02.952ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
52452025-08-26T21:50:02.952ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
52462025-08-26T21:50:02.952ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
52472025-08-26T21:50:02.952ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
52482025-08-26T21:50:02.952ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
52492025-08-26T21:50:02.957ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 94.93383ms iteration = 9 status = {"errors":[],"requests_created_ok":[]}
52502025-08-26T21:50:02.958ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
52512025-08-26T21:50:02.958ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
52522025-08-26T21:50:02.958ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
52532025-08-26T21:50:02.961ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
52542025-08-26T21:50:02.961ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
52552025-08-26T21:50:02.963ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
52562025-08-26T21:50:02.963ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
52572025-08-26T21:50:02.963ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
52582025-08-26T21:50:02.964ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
52592025-08-26T21:50:02.965ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
52602025-08-26T21:50:02.965ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 92.582447ms iteration = 25 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
52612025-08-26T21:50:02.968ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: dd0c655a-95c2-48fc-b4c3-9733ed4b0faa, lookup_type: ById(dd0c655a-95c2-48fc-b4c3-9733ed4b0faa) } result = Ok(())
52622025-08-26T21:50:02.968ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
52632025-08-26T21:50:02.969ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
52642025-08-26T21:50:02.971ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 110.284856ms iteration = 9 status = 0
52652025-08-26T21:50:02.977ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
52662025-08-26T21:50:02.980ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
52672025-08-26T21:50:02.980ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
52682025-08-26T21:50:02.980ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
52692025-08-26T21:50:02.980ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 119.239948ms iteration = 10 status = {"physical_disks_added":0}
52702025-08-26T21:50:02.982ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
52712025-08-26T21:50:02.983ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
52722025-08-26T21:50:02.983ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
52732025-08-26T21:50:02.983ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
52742025-08-26T21:50:02.983ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
52752025-08-26T21:50:02.983ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
52762025-08-26T21:50:02.985ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
52772025-08-26T21:50:02.985ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
52782025-08-26T21:50:02.985ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 124.766663ms iteration = 9 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
52792025-08-26T21:50:02.987ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
52802025-08-26T21:50:02.987ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 8e720f02-dde8-4423-a6e3-91698373805a dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
52812025-08-26T21:50:02.987ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
52822025-08-26T21:50:02.989ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
52832025-08-26T21:50:02.991ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
52842025-08-26T21:50:02.994ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
52852025-08-26T21:50:02.994ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:138 found = 0 not_ready_to_be_deleted = 0
52862025-08-26T21:50:02.994ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 133.431718ms iteration = 5 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
52872025-08-26T21:50:02.995ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
52882025-08-26T21:50:02.995ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
52892025-08-26T21:50:02.995ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
52902025-08-26T21:50:02.996ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 135.481484ms iteration = 25 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
52912025-08-26T21:50:02.998ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 8e720f02-dde8-4423-a6e3-91698373805a dns_zone_name = control-plane.oxide.internal found_so_far = 59 version = 1
52922025-08-26T21:50:02.998ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
52932025-08-26T21:50:02.998ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2025-08-26 21:46:04.065217 UTC dns_group = internal generation = 1
52942025-08-26T21:50:02.998ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 163.886154ms iteration = 5 status = {"generation":1}
52952025-08-26T21:50:03.000ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 138.335579ms iteration = 9 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
52962025-08-26T21:50:03.001ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
52972025-08-26T21:50:03.007ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
52982025-08-26T21:50:03.007ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
52992025-08-26T21:50:03.007ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
53002025-08-26T21:50:03.007ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
53012025-08-26T21:50:03.007ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 146.289687ms iteration = 9 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
53022025-08-26T21:50:03.021ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
53032025-08-26T21:50:03.021ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = dfb404ab-acb8-43de-afe8-b429af0c161d dns_zone_name = oxide.test found_so_far = 0 version = 2
53042025-08-26T21:50:03.021ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
53052025-08-26T21:50:03.021ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
53062025-08-26T21:50:03.023ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
53072025-08-26T21:50:03.026ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
53082025-08-26T21:50:03.026ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
53092025-08-26T21:50:03.026ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
53102025-08-26T21:50:03.028ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
53112025-08-26T21:50:03.029ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
53122025-08-26T21:50:03.029ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = dfb404ab-acb8-43de-afe8-b429af0c161d dns_zone_name = oxide.test found_so_far = 4 version = 2
53132025-08-26T21:50:03.029ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
53142025-08-26T21:50:03.029ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2025-08-26 21:46:05.296205 UTC dns_group = external generation = 2
53152025-08-26T21:50:03.029ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 194.924124ms iteration = 5 status = {"generation":2}
53162025-08-26T21:50:03.038ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 176.186482ms iteration = 9 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
53172025-08-26T21:50:03.043ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:407
53182025-08-26T21:50:03.044ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 183.049426ms iteration = 9 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
53192025-08-26T21:50:03.047ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-08-26T21:45:59.866217Z, time_modified: 2025-08-26T21:45:59.866217Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 9396aa55-de6f-4d6f-96db-d28e1c76fa9c, db_silo: Silo { identity: SiloIdentity { id: 9396aa55-de6f-4d6f-96db-d28e1c76fa9c, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-08-26T21:46:05.245521Z, time_modified: 2025-08-26T21:46:05.245521Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "e8529d720e4b0de099753dcdb373abb870af04baacd7c7dd850f5254858081af" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 9396aa55-de6f-4d6f-96db-d28e1c76fa9c, db_silo: Silo { identity: SiloIdentity { id: 9396aa55-de6f-4d6f-96db-d28e1c76fa9c, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-08-26T21:46:05.245521Z, time_modified: 2025-08-26T21:46:05.245521Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "e8529d720e4b0de099753dcdb373abb870af04baacd7c7dd850f5254858081af" }] }) }
53202025-08-26T21:50:03.047ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 213.392981ms iteration = 5 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"9396aa55-de6f-4d6f-96db-d28e1c76fa9c","tls_certs":["e8529d720e4b0de099753dcdb373abb870af04baacd7c7dd850f5254858081af"]}},"default_endpoint":{"silo_id":"9396aa55-de6f-4d6f-96db-d28e1c76fa9c","tls_certs":["e8529d720e4b0de099753dcdb373abb870af04baacd7c7dd850f5254858081af"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
53212025-08-26T21:50:03.053ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
53222025-08-26T21:50:03.054ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
53232025-08-26T21:50:03.058ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1213 zone_address = fd00:1122:3344:101::2
53242025-08-26T21:50:03.058ZDEBGnexus (MgsClient): client request background_task = nat_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
53252025-08-26T21:50:03.059ZDEBGnexus (MgsClient): client response background_task = nat_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "10bccb16-6812-4727-9bbb-130b1f8bf612", "content-length": "26", "date": "Tue, 26 Aug 2025 21:50:03 GMT"} })
53262025-08-26T21:50:03.059ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1216 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
53272025-08-26T21:50:03.060ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
53282025-08-26T21:50:03.084ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1213 zone_address = fd00:1122:3344:101::2
53292025-08-26T21:50:03.084ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
53302025-08-26T21:50:03.085ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "d84a30ff-079e-4842-84c3-7fa4fc299dde", "content-length": "26", "date": "Tue, 26 Aug 2025 21:50:03 GMT"} })
53312025-08-26T21:50:03.085ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1216 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
53322025-08-26T21:50:03.086ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
53332025-08-26T21:50:03.108ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1213 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 zone_address = fd00:1122:3344:101::2
53342025-08-26T21:50:03.108ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
53352025-08-26T21:50:03.108ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "2eaf0c29-880e-41aa-a9c5-b47a035c9522", "content-length": "26", "date": "Tue, 26 Aug 2025 21:50:03 GMT"} })
53362025-08-26T21:50:03.108ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1216 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
53372025-08-26T21:50:03.110ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
53382025-08-26T21:50:03.122ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
53392025-08-26T21:50:03.185ZINFOnexus (BackgroundTasks): instance reincarnation completed successfully background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:113 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0
53402025-08-26T21:50:03.185ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 324.605547ms iteration = 5 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
53412025-08-26T21:50:03.187ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
53422025-08-26T21:50:03.188ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
53432025-08-26T21:50:03.188ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
53442025-08-26T21:50:03.189ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
53452025-08-26T21:50:03.192ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
53462025-08-26T21:50:03.192ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
53472025-08-26T21:50:03.192ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
53482025-08-26T21:50:03.194ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
53492025-08-26T21:50:03.195ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
53502025-08-26T21:50:03.195ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
53512025-08-26T21:50:03.196ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
53522025-08-26T21:50:03.217ZINFOnexus (BackgroundTasks): VPC route manager sled dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
53532025-08-26T21:50:03.217ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
53542025-08-26T21:50:03.218ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "6357c044-ba41-4102-af07-dea8684d1c33", "content-length": "372", "date": "Tue, 26 Aug 2025 21:50:03 GMT"} })
53552025-08-26T21:50:03.218ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
53562025-08-26T21:50:03.218ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
53572025-08-26T21:50:03.228ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
53582025-08-26T21:50:03.229ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
53592025-08-26T21:50:03.230ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 396.454007ms iteration = 9 status = null
53602025-08-26T21:50:03.232ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
53612025-08-26T21:50:03.232ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
53622025-08-26T21:50:03.233ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "d57d9c53-f89a-45e8-87ac-090a3f989b68", "date": "Tue, 26 Aug 2025 21:50:03 GMT"} })
53632025-08-26T21:50:03.233ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
53642025-08-26T21:50:03.233ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
53652025-08-26T21:50:03.235ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
53662025-08-26T21:50:03.235ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
53672025-08-26T21:50:03.238ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
53682025-08-26T21:50:03.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
53692025-08-26T21:50:03.241ZINFOnexus (BackgroundTasks): VPC route manager sled dd0c655a-95c2-48fc-b4c3-9733ed4b0faa push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
53702025-08-26T21:50:03.241ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
53712025-08-26T21:50:03.242ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "ddbfc51e-9bdd-452d-a74a-cd9ceb62caa8", "date": "Tue, 26 Aug 2025 21:50:03 GMT"} })
53722025-08-26T21:50:03.243ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 382.705067ms iteration = 9 status = {}
53732025-08-26T21:50:03.267ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
53742025-08-26T21:50:03.267ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "4bfc7197-7b3b-4c97-b162-e156f86f3f3c", "date": "Tue, 26 Aug 2025 21:50:03 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
53752025-08-26T21:50:03.267ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
53762025-08-26T21:50:03.269ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "c13a9af7-8e05-4017-bfe0-da90efbb8972", "date": "Tue, 26 Aug 2025 21:50:03 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
53772025-08-26T21:50:03.269ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
53782025-08-26T21:50:03.271ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
53792025-08-26T21:50:03.271ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 437.449691ms iteration = 7 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
53802025-08-26T21:50:03.275ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 440.666634ms iteration = 9 status = {}
53812025-08-26T21:50:03.291ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
53822025-08-26T21:50:03.292ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "109c0be1-6ff1-43aa-a850-fa6af1ee4d5a", "date": "Tue, 26 Aug 2025 21:50:03 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
53832025-08-26T21:50:03.292ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
53842025-08-26T21:50:03.294ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "22330f82-0639-4840-8ae9-b7d67f1a5b5f", "date": "Tue, 26 Aug 2025 21:50:03 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
53852025-08-26T21:50:03.294ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
53862025-08-26T21:50:03.296ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "b2d049a1-cd38-4cc6-9adf-a6e02fa62961", "date": "Tue, 26 Aug 2025 21:50:03 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
53872025-08-26T21:50:03.296ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
53882025-08-26T21:50:03.298ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
53892025-08-26T21:50:03.298ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 464.071621ms iteration = 7 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
53902025-08-26T21:50:03.333ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
53912025-08-26T21:50:03.334ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "8e32a1c6-33b7-44cc-ad39-be3ff507750d", "content-length": "691", "date": "Tue, 26 Aug 2025 21:50:03 GMT"} })
53922025-08-26T21:50:03.334ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
53932025-08-26T21:50:03.334ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
53942025-08-26T21:50:03.334ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
53952025-08-26T21:50:03.334ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 473.840981ms iteration = 9 status = {}
53962025-08-26T21:50:03.360ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
53972025-08-26T21:50:03.361ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
53982025-08-26T21:50:03.363ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: d24c5c55-3a21-4a91-aeb4-45362e822df6, rack_id: e37a46f2-f789-4ca4-9841-f8c6dd705964, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(88d84f37-a4c8-4abc-af1f-27987b0d39d9) } rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0
53992025-08-26T21:50:03.363ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
54002025-08-26T21:50:03.363ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
54012025-08-26T21:50:03.376ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1832 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
54022025-08-26T21:50:03.376ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1859 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 88d84f37-a4c8-4abc-af1f-27987b0d39d9
54032025-08-26T21:50:03.376ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
54042025-08-26T21:50:03.376ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
54052025-08-26T21:50:03.377ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
54062025-08-26T21:50:03.377ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
54072025-08-26T21:50:03.377ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
54082025-08-26T21:50:03.377ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2006 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
54092025-08-26T21:50:03.377ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:408 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
54102025-08-26T21:50:03.377ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2042 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
54112025-08-26T21:50:03.377ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
54122025-08-26T21:50:03.377ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
54132025-08-26T21:50:03.377ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
54142025-08-26T21:50:03.379ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0
54152025-08-26T21:50:03.379ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
54162025-08-26T21:50:03.486ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "38248970-1ddc-4669-8e8f-83282207d92d", "date": "Tue, 26 Aug 2025 21:50:03 GMT"} })
54172025-08-26T21:50:03.486ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch = Switch0
54182025-08-26T21:50:03.486ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
54192025-08-26T21:50:03.487ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "32243574-24ac-4c1a-ae29-7562a0c8e2d6", "content-length": "682", "date": "Tue, 26 Aug 2025 21:50:03 GMT"} })
54202025-08-26T21:50:03.487ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
54212025-08-26T21:50:03.487ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
54222025-08-26T21:50:03.491ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
54232025-08-26T21:50:03.491ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
54242025-08-26T21:50:03.494ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
54252025-08-26T21:50:03.495ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
54262025-08-26T21:50:03.496ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
54272025-08-26T21:50:03.497ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
54282025-08-26T21:50:03.504ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 643.401124ms iteration = 9 status = {}
54292025-08-26T21:50:05.353ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 328 local_addr = [fd00:1122:3344:101::c]:42016 method = GET remote_addr = [fd00:1122:3344:101::d]:37550 req_id = 2ab51c70-7b7d-411c-84e4-cc7c2520fd8e response_code = 200 uri = /5d0959e5-47a6-4bf3-a52b-57950b9d900d
54302025-08-26T21:50:07.848ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 50 reason = Timeout
54312025-08-26T21:50:07.848ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
54322025-08-26T21:50:07.848ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
54332025-08-26T21:50:07.850ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54342025-08-26T21:50:07.853ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
54352025-08-26T21:50:07.853ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
54362025-08-26T21:50:07.853ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
54372025-08-26T21:50:07.854ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
54382025-08-26T21:50:07.854ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 6.246171ms iteration = 50 status = {"chicken_switches_updated":false}
54392025-08-26T21:50:12.847ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 51 reason = Timeout
54402025-08-26T21:50:12.847ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
54412025-08-26T21:50:12.848ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
54422025-08-26T21:50:12.850ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54432025-08-26T21:50:12.853ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
54442025-08-26T21:50:12.853ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
54452025-08-26T21:50:12.853ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
54462025-08-26T21:50:12.855ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
54472025-08-26T21:50:12.855ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.924618ms iteration = 51 status = {"chicken_switches_updated":false}
54482025-08-26T21:50:12.861ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 26 reason = Timeout
54492025-08-26T21:50:12.861ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
54502025-08-26T21:50:12.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
54512025-08-26T21:50:12.863ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
54522025-08-26T21:50:12.863ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
54532025-08-26T21:50:12.873ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 26 reason = Timeout
54542025-08-26T21:50:12.873ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
54552025-08-26T21:50:12.873ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
54562025-08-26T21:50:12.875ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54572025-08-26T21:50:12.878ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
54582025-08-26T21:50:12.878ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
54592025-08-26T21:50:12.878ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
54602025-08-26T21:50:12.880ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
54612025-08-26T21:50:12.880ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
54622025-08-26T21:50:12.882ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54632025-08-26T21:50:12.884ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
54642025-08-26T21:50:12.884ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
54652025-08-26T21:50:12.884ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
54662025-08-26T21:50:12.899ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 38.571599ms iteration = 26 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
54672025-08-26T21:50:12.902ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
54682025-08-26T21:50:12.902ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 29.166257ms iteration = 26 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
54692025-08-26T21:50:15.353ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 284 local_addr = [fd00:1122:3344:101::c]:42016 method = GET remote_addr = [fd00:1122:3344:101::d]:37550 req_id = fedd6989-9470-4295-afd0-ddc57f697410 response_code = 200 uri = /5d0959e5-47a6-4bf3-a52b-57950b9d900d
54702025-08-26T21:50:17.848ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 52 reason = Timeout
54712025-08-26T21:50:17.848ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
54722025-08-26T21:50:17.848ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
54732025-08-26T21:50:17.850ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54742025-08-26T21:50:17.853ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
54752025-08-26T21:50:17.853ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
54762025-08-26T21:50:17.853ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
54772025-08-26T21:50:17.854ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
54782025-08-26T21:50:17.854ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 6.306692ms iteration = 52 status = {"chicken_switches_updated":false}
54792025-08-26T21:50:22.847ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 53 reason = Timeout
54802025-08-26T21:50:22.847ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
54812025-08-26T21:50:22.848ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
54822025-08-26T21:50:22.850ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54832025-08-26T21:50:22.853ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
54842025-08-26T21:50:22.853ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
54852025-08-26T21:50:22.853ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
54862025-08-26T21:50:22.854ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
54872025-08-26T21:50:22.854ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.06882ms iteration = 53 status = {"chicken_switches_updated":false}
54882025-08-26T21:50:22.861ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 27 reason = Timeout
54892025-08-26T21:50:22.861ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
54902025-08-26T21:50:22.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
54912025-08-26T21:50:22.863ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
54922025-08-26T21:50:22.863ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
54932025-08-26T21:50:22.865ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.216175ms iteration = 27 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
54942025-08-26T21:50:22.873ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 27 reason = Timeout
54952025-08-26T21:50:22.873ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
54962025-08-26T21:50:22.873ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
54972025-08-26T21:50:22.875ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54982025-08-26T21:50:22.878ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
54992025-08-26T21:50:22.878ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
55002025-08-26T21:50:22.878ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
55012025-08-26T21:50:22.880ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
55022025-08-26T21:50:22.880ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
55032025-08-26T21:50:22.882ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
55042025-08-26T21:50:22.885ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
55052025-08-26T21:50:22.885ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
55062025-08-26T21:50:22.885ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
55072025-08-26T21:50:22.899ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
55082025-08-26T21:50:22.899ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 25.767731ms iteration = 27 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
55092025-08-26T21:50:25.353ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 312 local_addr = [fd00:1122:3344:101::c]:42016 method = GET remote_addr = [fd00:1122:3344:101::d]:37550 req_id = 7ce63952-d481-493b-8ac7-0f30e32fd9d7 response_code = 200 uri = /5d0959e5-47a6-4bf3-a52b-57950b9d900d
55102025-08-26T21:50:27.847ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 54 reason = Timeout
55112025-08-26T21:50:27.847ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
55122025-08-26T21:50:27.848ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
55132025-08-26T21:50:27.849ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
55142025-08-26T21:50:27.852ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
55152025-08-26T21:50:27.852ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
55162025-08-26T21:50:27.852ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
55172025-08-26T21:50:27.853ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
55182025-08-26T21:50:27.853ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 6.086367ms iteration = 54 status = {"chicken_switches_updated":false}
55192025-08-26T21:50:32.833ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 10 reason = Timeout
55202025-08-26T21:50:32.833ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 10 reason = Timeout
55212025-08-26T21:50:32.833ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
55222025-08-26T21:50:32.833ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 10 reason = Timeout
55232025-08-26T21:50:32.833ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
55242025-08-26T21:50:32.834ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
55252025-08-26T21:50:32.834ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
55262025-08-26T21:50:32.836ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1136, tv_nsec: 694313827 } })
55272025-08-26T21:50:32.836ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1136, tv_nsec: 694442850 } })
55282025-08-26T21:50:32.836ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 2.967196ms iteration = 10 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
55292025-08-26T21:50:32.836ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1205
55302025-08-26T21:50:32.836ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1205
55312025-08-26T21:50:32.848ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 55 reason = Timeout
55322025-08-26T21:50:32.848ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
55332025-08-26T21:50:32.848ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
55342025-08-26T21:50:32.849ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
55352025-08-26T21:50:32.852ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
55362025-08-26T21:50:32.852ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
55372025-08-26T21:50:32.852ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
55382025-08-26T21:50:32.853ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
55392025-08-26T21:50:32.853ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 5.656118ms iteration = 55 status = {"chicken_switches_updated":false}
55402025-08-26T21:50:32.861ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 10 reason = Timeout
55412025-08-26T21:50:32.861ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
55422025-08-26T21:50:32.861ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 10 reason = Timeout
55432025-08-26T21:50:32.861ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 11 reason = Timeout
55442025-08-26T21:50:32.861ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
55452025-08-26T21:50:32.861ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
55462025-08-26T21:50:32.861ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 10 reason = Timeout
55472025-08-26T21:50:32.861ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 10 reason = Timeout
55482025-08-26T21:50:32.861ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
55492025-08-26T21:50:32.861ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
55502025-08-26T21:50:32.861ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 10 reason = Timeout
55512025-08-26T21:50:32.861ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
55522025-08-26T21:50:32.861ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 28 reason = Timeout
55532025-08-26T21:50:32.861ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
55542025-08-26T21:50:32.861ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
55552025-08-26T21:50:32.861ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 10 reason = Timeout
55562025-08-26T21:50:32.861ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
55572025-08-26T21:50:32.861ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
55582025-08-26T21:50:32.861ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 10 reason = Timeout
55592025-08-26T21:50:32.861ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
55602025-08-26T21:50:32.861ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 10 reason = Timeout
55612025-08-26T21:50:32.861ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
55622025-08-26T21:50:32.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
55632025-08-26T21:50:32.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
55642025-08-26T21:50:32.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
55652025-08-26T21:50:32.861ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 10 reason = Timeout
55662025-08-26T21:50:32.861ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 10 reason = Timeout
55672025-08-26T21:50:32.861ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
55682025-08-26T21:50:32.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
55692025-08-26T21:50:32.861ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 10 reason = Timeout
55702025-08-26T21:50:32.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
55712025-08-26T21:50:32.861ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
55722025-08-26T21:50:32.861ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 10 reason = Timeout
55732025-08-26T21:50:32.861ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 6.771\u{b5}s iteration = 10 status = {"disabled":true,"errors":[],"sps":[]}
55742025-08-26T21:50:32.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
55752025-08-26T21:50:32.861ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 10 reason = Timeout
55762025-08-26T21:50:32.861ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
55772025-08-26T21:50:32.861ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 10 reason = Timeout
55782025-08-26T21:50:32.861ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
55792025-08-26T21:50:32.861ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
55802025-08-26T21:50:32.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
55812025-08-26T21:50:32.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
55822025-08-26T21:50:32.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
55832025-08-26T21:50:32.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
55842025-08-26T21:50:32.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
55852025-08-26T21:50:32.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
55862025-08-26T21:50:32.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
55872025-08-26T21:50:32.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
55882025-08-26T21:50:32.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
55892025-08-26T21:50:32.862ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
55902025-08-26T21:50:32.862ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
55912025-08-26T21:50:32.863ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
55922025-08-26T21:50:32.863ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
55932025-08-26T21:50:32.863ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
55942025-08-26T21:50:32.863ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
55952025-08-26T21:50:32.863ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
55962025-08-26T21:50:32.863ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
55972025-08-26T21:50:32.864ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 2.556708ms iteration = 10 status = {"errors":[],"finish_invoked_ok":[]}
55982025-08-26T21:50:32.867ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
55992025-08-26T21:50:32.867ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
56002025-08-26T21:50:32.867ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
56012025-08-26T21:50:32.867ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
56022025-08-26T21:50:32.867ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
56032025-08-26T21:50:32.867ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
56042025-08-26T21:50:32.867ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
56052025-08-26T21:50:32.867ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
56062025-08-26T21:50:32.867ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
56072025-08-26T21:50:32.867ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
56082025-08-26T21:50:32.867ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
56092025-08-26T21:50:32.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
56102025-08-26T21:50:32.868ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
56112025-08-26T21:50:32.868ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
56122025-08-26T21:50:32.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
56132025-08-26T21:50:32.869ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
56142025-08-26T21:50:32.869ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
56152025-08-26T21:50:32.869ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
56162025-08-26T21:50:32.869ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
56172025-08-26T21:50:32.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
56182025-08-26T21:50:32.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
56192025-08-26T21:50:32.870ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
56202025-08-26T21:50:32.871ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
56212025-08-26T21:50:32.871ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
56222025-08-26T21:50:32.871ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
56232025-08-26T21:50:32.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
56242025-08-26T21:50:32.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
56252025-08-26T21:50:32.871ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1136, tv_nsec: 729723755 } })
56262025-08-26T21:50:32.872ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1205 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
56272025-08-26T21:50:32.872ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 28 reason = Timeout
56282025-08-26T21:50:32.872ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
56292025-08-26T21:50:32.872ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
56302025-08-26T21:50:32.874ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
56312025-08-26T21:50:32.874ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
56322025-08-26T21:50:32.874ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
56332025-08-26T21:50:32.874ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
56342025-08-26T21:50:32.874ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
56352025-08-26T21:50:32.874ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
56362025-08-26T21:50:32.875ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:550 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
56372025-08-26T21:50:32.875ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 14.209471ms iteration = 10 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
56382025-08-26T21:50:32.877ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
56392025-08-26T21:50:32.877ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
56402025-08-26T21:50:32.877ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
56412025-08-26T21:50:32.881ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
56422025-08-26T21:50:32.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
56432025-08-26T21:50:32.884ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
56442025-08-26T21:50:32.887ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
56452025-08-26T21:50:32.887ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
56462025-08-26T21:50:32.887ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
56472025-08-26T21:50:32.913ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
56482025-08-26T21:50:32.913ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
56492025-08-26T21:50:32.918ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 57.053086ms iteration = 10 status = {"errors":[],"requests_created_ok":[]}
56502025-08-26T21:50:32.918ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
56512025-08-26T21:50:32.918ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
56522025-08-26T21:50:32.919ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
56532025-08-26T21:50:32.919ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 46.679493ms iteration = 28 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
56542025-08-26T21:50:32.919ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
56552025-08-26T21:50:32.919ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
56562025-08-26T21:50:32.919ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
56572025-08-26T21:50:32.920ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
56582025-08-26T21:50:32.921ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 60.113515ms iteration = 10 status = {"errors":[],"garbage_collect_requested":[]}
56592025-08-26T21:50:32.922ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 61.290242ms iteration = 28 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
56602025-08-26T21:50:32.925ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
56612025-08-26T21:50:32.926ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
56622025-08-26T21:50:32.926ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
56632025-08-26T21:50:32.926ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
56642025-08-26T21:50:32.926ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1213 zone_address = fd00:1122:3344:101::2
56652025-08-26T21:50:32.926ZDEBGnexus (MgsClient): client request background_task = nat_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
56662025-08-26T21:50:32.926ZDEBGnexus (MgsClient): client response background_task = nat_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "5194ba9c-6a9a-4ece-ac5c-5b33b285f0b8", "content-length": "26", "date": "Tue, 26 Aug 2025 21:50:32 GMT"} })
56672025-08-26T21:50:32.926ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1216 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
56682025-08-26T21:50:32.928ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1213 zone_address = fd00:1122:3344:101::2
56692025-08-26T21:50:32.928ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
56702025-08-26T21:50:32.928ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "fe162126-e466-4a69-8840-cd47877aa0ad", "content-length": "26", "date": "Tue, 26 Aug 2025 21:50:32 GMT"} })
56712025-08-26T21:50:32.928ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1216 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
56722025-08-26T21:50:32.928ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
56732025-08-26T21:50:32.930ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
56742025-08-26T21:50:32.930ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
56752025-08-26T21:50:32.931ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
56762025-08-26T21:50:32.932ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
56772025-08-26T21:50:32.934ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
56782025-08-26T21:50:32.934ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
56792025-08-26T21:50:32.934ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
56802025-08-26T21:50:32.937ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
56812025-08-26T21:50:32.938ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
56822025-08-26T21:50:32.938ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
56832025-08-26T21:50:32.940ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
56842025-08-26T21:50:32.940ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
56852025-08-26T21:50:32.941ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 80.584386ms iteration = 11 status = {"physical_disks_added":0}
56862025-08-26T21:50:32.941ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
56872025-08-26T21:50:32.941ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
56882025-08-26T21:50:32.941ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
56892025-08-26T21:50:32.942ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
56902025-08-26T21:50:32.944ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: dd0c655a-95c2-48fc-b4c3-9733ed4b0faa, lookup_type: ById(dd0c655a-95c2-48fc-b4c3-9733ed4b0faa) } result = Ok(())
56912025-08-26T21:50:32.944ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
56922025-08-26T21:50:32.945ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
56932025-08-26T21:50:32.946ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 85.559819ms iteration = 10 status = 0
56942025-08-26T21:50:32.961ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
56952025-08-26T21:50:32.961ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
56962025-08-26T21:50:32.961ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
56972025-08-26T21:50:32.961ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
56982025-08-26T21:50:32.965ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 103.942663ms iteration = 10 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
56992025-08-26T21:50:32.971ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 109.991379ms iteration = 10 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
57002025-08-26T21:50:32.971ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
57012025-08-26T21:50:32.971ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
57022025-08-26T21:50:32.973ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
57032025-08-26T21:50:32.975ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
57042025-08-26T21:50:32.975ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
57052025-08-26T21:50:32.975ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
57062025-08-26T21:50:32.976ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1213 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 zone_address = fd00:1122:3344:101::2
57072025-08-26T21:50:32.976ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
57082025-08-26T21:50:32.976ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "8e64baef-d85a-44f1-835c-86703826c863", "content-length": "26", "date": "Tue, 26 Aug 2025 21:50:32 GMT"} })
57092025-08-26T21:50:32.976ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1216 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
57102025-08-26T21:50:32.978ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
57112025-08-26T21:50:32.999ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
57122025-08-26T21:50:32.999ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
57132025-08-26T21:50:32.999ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 138.548124ms iteration = 10 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
57142025-08-26T21:50:33.013ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 151.551646ms iteration = 10 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
57152025-08-26T21:50:33.015ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:407
57162025-08-26T21:50:33.015ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 154.518483ms iteration = 10 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
57172025-08-26T21:50:33.059ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
57182025-08-26T21:50:33.059ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
57192025-08-26T21:50:33.062ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 228.284736ms iteration = 10 status = null
57202025-08-26T21:50:33.064ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 230.459756ms iteration = 10 status = {}
57212025-08-26T21:50:33.144ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
57222025-08-26T21:50:33.145ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "a056768d-0952-41c8-be03-62cd5f956a9e", "content-length": "691", "date": "Tue, 26 Aug 2025 21:50:33 GMT"} })
57232025-08-26T21:50:33.145ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
57242025-08-26T21:50:33.145ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
57252025-08-26T21:50:33.145ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
57262025-08-26T21:50:33.147ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 286.314734ms iteration = 10 status = {}
57272025-08-26T21:50:33.168ZINFOnexus (BackgroundTasks): VPC route manager sled dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
57282025-08-26T21:50:33.168ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
57292025-08-26T21:50:33.169ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "1b5faaad-257c-420f-b755-9437bdc52eb1", "content-length": "372", "date": "Tue, 26 Aug 2025 21:50:33 GMT"} })
57302025-08-26T21:50:33.169ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
57312025-08-26T21:50:33.169ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
57322025-08-26T21:50:33.183ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
57332025-08-26T21:50:33.183ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
57342025-08-26T21:50:33.183ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "beccdca8-60b5-4c6c-9686-14a06530dcda", "date": "Tue, 26 Aug 2025 21:50:33 GMT"} })
57352025-08-26T21:50:33.183ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
57362025-08-26T21:50:33.184ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
57372025-08-26T21:50:33.186ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
57382025-08-26T21:50:33.186ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
57392025-08-26T21:50:33.189ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
57402025-08-26T21:50:33.189ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
57412025-08-26T21:50:33.192ZINFOnexus (BackgroundTasks): VPC route manager sled dd0c655a-95c2-48fc-b4c3-9733ed4b0faa push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
57422025-08-26T21:50:33.192ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
57432025-08-26T21:50:33.193ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "b846ec98-dbba-454e-9687-80f7a6ce9200", "date": "Tue, 26 Aug 2025 21:50:33 GMT"} })
57442025-08-26T21:50:33.194ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 333.777944ms iteration = 10 status = {}
57452025-08-26T21:50:33.206ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
57462025-08-26T21:50:33.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
57472025-08-26T21:50:33.209ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: d24c5c55-3a21-4a91-aeb4-45362e822df6, rack_id: e37a46f2-f789-4ca4-9841-f8c6dd705964, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(88d84f37-a4c8-4abc-af1f-27987b0d39d9) } rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0
57482025-08-26T21:50:33.209ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
57492025-08-26T21:50:33.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
57502025-08-26T21:50:33.221ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1832 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
57512025-08-26T21:50:33.221ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1859 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 88d84f37-a4c8-4abc-af1f-27987b0d39d9
57522025-08-26T21:50:33.221ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
57532025-08-26T21:50:33.221ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
57542025-08-26T21:50:33.221ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
57552025-08-26T21:50:33.221ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
57562025-08-26T21:50:33.221ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
57572025-08-26T21:50:33.221ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2006 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
57582025-08-26T21:50:33.221ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:408 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
57592025-08-26T21:50:33.221ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2042 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
57602025-08-26T21:50:33.222ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
57612025-08-26T21:50:33.222ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
57622025-08-26T21:50:33.222ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
57632025-08-26T21:50:33.224ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0
57642025-08-26T21:50:33.224ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
57652025-08-26T21:50:33.328ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "7f236254-b9fc-4673-93b2-fab268ca8ffb", "date": "Tue, 26 Aug 2025 21:50:33 GMT"} })
57662025-08-26T21:50:33.328ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch = Switch0
57672025-08-26T21:50:33.328ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
57682025-08-26T21:50:33.329ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "380c43d8-3724-46ed-ac30-9a68215fcedb", "content-length": "682", "date": "Tue, 26 Aug 2025 21:50:33 GMT"} })
57692025-08-26T21:50:33.329ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
57702025-08-26T21:50:33.329ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
57712025-08-26T21:50:33.332ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
57722025-08-26T21:50:33.332ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
57732025-08-26T21:50:33.335ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
57742025-08-26T21:50:33.336ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
57752025-08-26T21:50:33.337ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
57762025-08-26T21:50:33.337ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
57772025-08-26T21:50:33.345ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 484.232425ms iteration = 10 status = {}
57782025-08-26T21:50:35.352ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 270 local_addr = [fd00:1122:3344:101::c]:42016 method = GET remote_addr = [fd00:1122:3344:101::d]:37550 req_id = bc49680d-589e-4eb2-bb10-cfcd1ef1d825 response_code = 200 uri = /5d0959e5-47a6-4bf3-a52b-57950b9d900d
57792025-08-26T21:50:37.847ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 56 reason = Timeout
57802025-08-26T21:50:37.847ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
57812025-08-26T21:50:37.848ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
57822025-08-26T21:50:37.850ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
57832025-08-26T21:50:37.853ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
57842025-08-26T21:50:37.853ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
57852025-08-26T21:50:37.853ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
57862025-08-26T21:50:37.855ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
57872025-08-26T21:50:37.855ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.579691ms iteration = 56 status = {"chicken_switches_updated":false}
57882025-08-26T21:50:42.848ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 57 reason = Timeout
57892025-08-26T21:50:42.848ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
57902025-08-26T21:50:42.849ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
57912025-08-26T21:50:42.850ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
57922025-08-26T21:50:42.853ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
57932025-08-26T21:50:42.853ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
57942025-08-26T21:50:42.854ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
57952025-08-26T21:50:42.856ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
57962025-08-26T21:50:42.856ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.560031ms iteration = 57 status = {"chicken_switches_updated":false}
57972025-08-26T21:50:42.861ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 29 reason = Timeout
57982025-08-26T21:50:42.861ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
57992025-08-26T21:50:42.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
58002025-08-26T21:50:42.863ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
58012025-08-26T21:50:42.863ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
58022025-08-26T21:50:42.872ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 29 reason = Timeout
58032025-08-26T21:50:42.872ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
58042025-08-26T21:50:42.873ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
58052025-08-26T21:50:42.874ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
58062025-08-26T21:50:42.877ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
58072025-08-26T21:50:42.877ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
58082025-08-26T21:50:42.877ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
58092025-08-26T21:50:42.879ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
58102025-08-26T21:50:42.879ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
58112025-08-26T21:50:42.881ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
58122025-08-26T21:50:42.883ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
58132025-08-26T21:50:42.883ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
58142025-08-26T21:50:42.884ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
58152025-08-26T21:50:42.901ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 40.018742ms iteration = 29 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
58162025-08-26T21:50:42.904ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
58172025-08-26T21:50:42.904ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 31.471219ms iteration = 29 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
58182025-08-26T21:50:45.353ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 329 local_addr = [fd00:1122:3344:101::c]:42016 method = GET remote_addr = [fd00:1122:3344:101::d]:37550 req_id = 40e3cfc5-f4cd-4657-93f6-6753fa51644f response_code = 200 uri = /5d0959e5-47a6-4bf3-a52b-57950b9d900d
58192025-08-26T21:50:47.848ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 58 reason = Timeout
58202025-08-26T21:50:47.848ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
58212025-08-26T21:50:47.848ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
58222025-08-26T21:50:47.850ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
58232025-08-26T21:50:47.853ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
58242025-08-26T21:50:47.853ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
58252025-08-26T21:50:47.853ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
58262025-08-26T21:50:47.855ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
58272025-08-26T21:50:47.855ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.250553ms iteration = 58 status = {"chicken_switches_updated":false}
58282025-08-26T21:50:52.847ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 59 reason = Timeout
58292025-08-26T21:50:52.847ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
58302025-08-26T21:50:52.848ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
58312025-08-26T21:50:52.849ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
58322025-08-26T21:50:52.852ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
58332025-08-26T21:50:52.852ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
58342025-08-26T21:50:52.852ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
58352025-08-26T21:50:52.854ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
58362025-08-26T21:50:52.854ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 6.985747ms iteration = 59 status = {"chicken_switches_updated":false}
58372025-08-26T21:50:52.860ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 30 reason = Timeout
58382025-08-26T21:50:52.860ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
58392025-08-26T21:50:52.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
58402025-08-26T21:50:52.862ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
58412025-08-26T21:50:52.863ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
58422025-08-26T21:50:52.864ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.483429ms iteration = 30 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
58432025-08-26T21:50:52.873ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 30 reason = Timeout
58442025-08-26T21:50:52.873ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
58452025-08-26T21:50:52.873ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
58462025-08-26T21:50:52.875ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
58472025-08-26T21:50:52.877ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
58482025-08-26T21:50:52.877ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
58492025-08-26T21:50:52.878ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
58502025-08-26T21:50:52.879ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
58512025-08-26T21:50:52.880ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
58522025-08-26T21:50:52.881ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
58532025-08-26T21:50:52.884ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
58542025-08-26T21:50:52.884ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
58552025-08-26T21:50:52.884ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
58562025-08-26T21:50:52.896ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
58572025-08-26T21:50:52.896ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 23.663663ms iteration = 30 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
58582025-08-26T21:50:55.353ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 423 local_addr = [fd00:1122:3344:101::c]:42016 method = GET remote_addr = [fd00:1122:3344:101::d]:37550 req_id = 6be376ef-cbc7-441c-9839-edfe71b6b33c response_code = 200 uri = /5d0959e5-47a6-4bf3-a52b-57950b9d900d
58592025-08-26T21:50:57.848ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 60 reason = Timeout
58602025-08-26T21:50:57.848ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
58612025-08-26T21:50:57.848ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
58622025-08-26T21:50:57.850ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
58632025-08-26T21:50:57.853ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
58642025-08-26T21:50:57.853ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
58652025-08-26T21:50:57.853ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
58662025-08-26T21:50:57.855ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
58672025-08-26T21:50:57.855ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.383187ms iteration = 60 status = {"chicken_switches_updated":false}
58682025-08-26T21:51:02.834ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 6 reason = Timeout
58692025-08-26T21:51:02.834ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 6 reason = Timeout
58702025-08-26T21:51:02.834ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 6 reason = Timeout
58712025-08-26T21:51:02.834ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
58722025-08-26T21:51:02.834ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 6 reason = Timeout
58732025-08-26T21:51:02.834ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-08-26 21:41:02.834465976 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
58742025-08-26T21:51:02.834ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
58752025-08-26T21:51:02.834ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
58762025-08-26T21:51:02.834ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 6 reason = Timeout
58772025-08-26T21:51:02.834ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
58782025-08-26T21:51:02.834ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
58792025-08-26T21:51:02.834ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
58802025-08-26T21:51:02.834ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 6 reason = Timeout
58812025-08-26T21:51:02.834ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 11 reason = Timeout
58822025-08-26T21:51:02.834ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 11 reason = Timeout
58832025-08-26T21:51:02.834ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
58842025-08-26T21:51:02.834ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 8 reason = Timeout
58852025-08-26T21:51:02.834ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 8 reason = Timeout
58862025-08-26T21:51:02.834ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 11 reason = Timeout
58872025-08-26T21:51:02.834ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
58882025-08-26T21:51:02.834ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
58892025-08-26T21:51:02.834ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
58902025-08-26T21:51:02.834ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
58912025-08-26T21:51:02.834ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("60abf731-31e1-4111-a3e2-65da89eaed87.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("84c849eb-0949-41d9-ad02-9a5509144564.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("84c849eb-0949-41d9-ad02-9a5509144564.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::8)) }], valid_until: Instant { tv_sec: 1166, tv_nsec: 692835803 } })
58922025-08-26T21:51:02.834ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
58932025-08-26T21:51:02.834ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
58942025-08-26T21:51:02.834ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
58952025-08-26T21:51:02.835ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("852fd1a3-de9a-47e8-a667-5e8dab880ad3.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("c68f1501-294c-4874-92ca-8fc0546a5870.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("d741a72f-8745-4723-9e80-e1e72987266b.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("d741a72f-8745-4723-9e80-e1e72987266b.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:3::1)) }], valid_until: Instant { tv_sec: 1166, tv_nsec: 692965306 } })
58962025-08-26T21:51:02.835ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
58972025-08-26T21:51:02.835ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 941.901\u{b5}s iteration = 6 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
58982025-08-26T21:51:02.835ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } dns_group = internal
58992025-08-26T21:51:02.835ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.065604ms iteration = 6 status = {"addresses":["[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:3::1]:5353"]}
59002025-08-26T21:51:02.835ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
59012025-08-26T21:51:02.836ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
59022025-08-26T21:51:02.836ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
59032025-08-26T21:51:02.836ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
59042025-08-26T21:51:02.836ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 7 reason = Timeout
59052025-08-26T21:51:02.836ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:91 reason = blueprint disabled target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8
59062025-08-26T21:51:02.836ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 13.49\u{b5}s iteration = 7 status = {"enabled":false,"target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8"}
59072025-08-26T21:51:02.836ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
59082025-08-26T21:51:02.837ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
59092025-08-26T21:51:02.837ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1166, tv_nsec: 695493813 } })
59102025-08-26T21:51:02.837ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1166, tv_nsec: 695619946 } })
59112025-08-26T21:51:02.837ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1205
59122025-08-26T21:51:02.837ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-08-26 21:41:02.834465976 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
59132025-08-26T21:51:02.837ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 3.406397ms iteration = 6 status = {"expiration":"2025-08-26T21:41:02.834465976Z","pruned":[]}
59142025-08-26T21:51:02.837ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1205
59152025-08-26T21:51:02.837ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
59162025-08-26T21:51:02.839ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
59172025-08-26T21:51:02.839ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
59182025-08-26T21:51:02.839ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
59192025-08-26T21:51:02.841ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
59202025-08-26T21:51:02.841ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
59212025-08-26T21:51:02.842ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
59222025-08-26T21:51:02.842ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
59232025-08-26T21:51:02.843ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
59242025-08-26T21:51:02.843ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
59252025-08-26T21:51:02.843ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
59262025-08-26T21:51:02.843ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
59272025-08-26T21:51:02.843ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
59282025-08-26T21:51:02.843ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
59292025-08-26T21:51:02.844ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
59302025-08-26T21:51:02.846ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 12.058892ms iteration = 11 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
59312025-08-26T21:51:02.847ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
59322025-08-26T21:51:02.847ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
59332025-08-26T21:51:02.847ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
59342025-08-26T21:51:02.847ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 61 reason = Timeout
59352025-08-26T21:51:02.847ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
59362025-08-26T21:51:02.847ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
59372025-08-26T21:51:02.848ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
59382025-08-26T21:51:02.848ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
59392025-08-26T21:51:02.848ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
59402025-08-26T21:51:02.849ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
59412025-08-26T21:51:02.849ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
59422025-08-26T21:51:02.849ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
59432025-08-26T21:51:02.850ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
59442025-08-26T21:51:02.852ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
59452025-08-26T21:51:02.852ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
59462025-08-26T21:51:02.852ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
59472025-08-26T21:51:02.852ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
59482025-08-26T21:51:02.852ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
59492025-08-26T21:51:02.853ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
59502025-08-26T21:51:02.854ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
59512025-08-26T21:51:02.854ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 6.946287ms iteration = 61 status = {"chicken_switches_updated":false}
59522025-08-26T21:51:02.861ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_rendezvous iteration = 3 reason = Timeout
59532025-08-26T21:51:02.861ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 11 reason = Timeout
59542025-08-26T21:51:02.861ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 11 reason = Timeout
59552025-08-26T21:51:02.861ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
59562025-08-26T21:51:02.861ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
59572025-08-26T21:51:02.861ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 11 reason = Timeout
59582025-08-26T21:51:02.861ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
59592025-08-26T21:51:02.861ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 6 reason = Timeout
59602025-08-26T21:51:02.861ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:53
59612025-08-26T21:51:02.861ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
59622025-08-26T21:51:02.861ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 11 reason = Timeout
59632025-08-26T21:51:02.861ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
59642025-08-26T21:51:02.861ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 11 reason = Timeout
59652025-08-26T21:51:02.861ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
59662025-08-26T21:51:02.861ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 8 reason = Timeout
59672025-08-26T21:51:02.861ZDEBGnexus (BackgroundTasks): blueprint planning disabled, doing nothing background_task = blueprint_planner
59682025-08-26T21:51:02.861ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 15.59\u{b5}s iteration = 8 status = "Disabled"
59692025-08-26T21:51:02.861ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 6 reason = Timeout
59702025-08-26T21:51:02.861ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
59712025-08-26T21:51:02.861ZDEBGnexus (BackgroundTasks): activating background_task = service_firewall_rule_propagation iteration = 2 reason = Timeout
59722025-08-26T21:51:02.861ZDEBGnexus (service-firewall-rule-progator): starting background task for service firewall rule propagation background_task = service_firewall_rule_propagation
59732025-08-26T21:51:02.861ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 11 reason = Timeout
59742025-08-26T21:51:02.861ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
59752025-08-26T21:51:02.861ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 11 reason = Timeout
59762025-08-26T21:51:02.861ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
59772025-08-26T21:51:02.861ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
59782025-08-26T21:51:02.861ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
59792025-08-26T21:51:02.861ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 12 reason = Timeout
59802025-08-26T21:51:02.861ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
59812025-08-26T21:51:02.861ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 11 reason = Timeout
59822025-08-26T21:51:02.861ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
59832025-08-26T21:51:02.861ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 6 reason = Timeout
59842025-08-26T21:51:02.861ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
59852025-08-26T21:51:02.861ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 31 reason = Timeout
59862025-08-26T21:51:02.861ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
59872025-08-26T21:51:02.861ZDEBGnexus (BackgroundTasks): activating background_task = tuf_artifact_replication iteration = 2 reason = Timeout
59882025-08-26T21:51:02.861ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
59892025-08-26T21:51:02.861ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 11 reason = Timeout
59902025-08-26T21:51:02.861ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
59912025-08-26T21:51:02.861ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 11 reason = Timeout
59922025-08-26T21:51:02.861ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
59932025-08-26T21:51:02.861ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
59942025-08-26T21:51:02.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
59952025-08-26T21:51:02.861ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 11 reason = Timeout
59962025-08-26T21:51:02.861ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
59972025-08-26T21:51:02.861ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 11 reason = Timeout
59982025-08-26T21:51:02.861ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
59992025-08-26T21:51:02.861ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 11 reason = Timeout
60002025-08-26T21:51:02.861ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
60012025-08-26T21:51:02.861ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 6 reason = Timeout
60022025-08-26T21:51:02.861ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
60032025-08-26T21:51:02.861ZDEBGnexus (BackgroundTasks): activating background_task = alert_dispatcher iteration = 6 reason = Timeout
60042025-08-26T21:51:02.861ZDEBGnexus (BackgroundTasks): activating background_task = webhook_deliverator iteration = 6 reason = Timeout
60052025-08-26T21:51:02.861ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
60062025-08-26T21:51:02.861ZDEBGnexus (BackgroundTasks): roles background_task = webhook_deliverator roles = RoleSet { roles: {} }
60072025-08-26T21:51:02.861ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 11 reason = Timeout
60082025-08-26T21:51:02.861ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 11.09\u{b5}s iteration = 11 status = {"disabled":true,"errors":[],"sps":[]}
60092025-08-26T21:51:02.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
60102025-08-26T21:51:02.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
60112025-08-26T21:51:02.861ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
60122025-08-26T21:51:02.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
60132025-08-26T21:51:02.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
60142025-08-26T21:51:02.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
60152025-08-26T21:51:02.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
60162025-08-26T21:51:02.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
60172025-08-26T21:51:02.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
60182025-08-26T21:51:02.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
60192025-08-26T21:51:02.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
60202025-08-26T21:51:02.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
60212025-08-26T21:51:02.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = alert_dispatcher resource = Database result = Ok(())
60222025-08-26T21:51:02.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
60232025-08-26T21:51:02.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
60242025-08-26T21:51:02.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
60252025-08-26T21:51:02.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
60262025-08-26T21:51:02.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = webhook_deliverator resource = Database result = Ok(())
60272025-08-26T21:51:02.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
60282025-08-26T21:51:02.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
60292025-08-26T21:51:02.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
60302025-08-26T21:51:02.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
60312025-08-26T21:51:02.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
60322025-08-26T21:51:02.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
60332025-08-26T21:51:02.863ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
60342025-08-26T21:51:02.863ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
60352025-08-26T21:51:02.864ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
60362025-08-26T21:51:02.864ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
60372025-08-26T21:51:02.865ZINFOnexus (BackgroundTasks): all webhook delivery tasks completed background_task = webhook_deliverator file = nexus/src/app/background/tasks/webhook_deliverator.rs:204 num_receivers = 0
60382025-08-26T21:51:02.865ZDEBGnexus (BackgroundTasks): activation complete background_task = webhook_deliverator elapsed = 3.611461ms iteration = 6 status = {"by_rx":{},"error":null}
60392025-08-26T21:51:02.865ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 3.747115ms iteration = 6 status = {"errors":[],"found_port_ok":[]}
60402025-08-26T21:51:02.865ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
60412025-08-26T21:51:02.865ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
60422025-08-26T21:51:02.865ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
60432025-08-26T21:51:02.865ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
60442025-08-26T21:51:02.866ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 4.816169ms iteration = 11 status = {"errors":[],"finish_invoked_ok":[]}
60452025-08-26T21:51:02.866ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
60462025-08-26T21:51:02.866ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
60472025-08-26T21:51:02.868ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
60482025-08-26T21:51:02.868ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
60492025-08-26T21:51:02.868ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
60502025-08-26T21:51:02.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
60512025-08-26T21:51:02.869ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
60522025-08-26T21:51:02.869ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
60532025-08-26T21:51:02.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
60542025-08-26T21:51:02.869ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
60552025-08-26T21:51:02.869ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
60562025-08-26T21:51:02.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
60572025-08-26T21:51:02.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
60582025-08-26T21:51:02.869ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Fleet result = Ok(())
60592025-08-26T21:51:02.869ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
60602025-08-26T21:51:02.870ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
60612025-08-26T21:51:02.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
60622025-08-26T21:51:02.870ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
60632025-08-26T21:51:02.870ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
60642025-08-26T21:51:02.870ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
60652025-08-26T21:51:02.870ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
60662025-08-26T21:51:02.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = alert_dispatcher resource = Database result = Ok(())
60672025-08-26T21:51:02.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
60682025-08-26T21:51:02.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
60692025-08-26T21:51:02.870ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
60702025-08-26T21:51:02.871ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
60712025-08-26T21:51:02.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
60722025-08-26T21:51:02.871ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 10.008735ms iteration = 11 status = {"errors":[],"requests_created_ok":[]}
60732025-08-26T21:51:02.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
60742025-08-26T21:51:02.873ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 31 reason = Timeout
60752025-08-26T21:51:02.873ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
60762025-08-26T21:51:02.873ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
60772025-08-26T21:51:02.873ZDEBGnexus (BackgroundTasks): activation complete background_task = alert_dispatcher elapsed = 11.688153ms iteration = 6 status = {"dispatched":[],"errors":[],"glob_version":"183.0.0","globs_reprocessed":{},"no_receivers":[]}
60782025-08-26T21:51:02.873ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
60792025-08-26T21:51:02.873ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
60802025-08-26T21:51:02.874ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
60812025-08-26T21:51:02.874ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
60822025-08-26T21:51:02.876ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
60832025-08-26T21:51:02.876ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
60842025-08-26T21:51:02.876ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Fleet result = Ok(())
60852025-08-26T21:51:02.876ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
60862025-08-26T21:51:02.876ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
60872025-08-26T21:51:02.877ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
60882025-08-26T21:51:02.877ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
60892025-08-26T21:51:02.878ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
60902025-08-26T21:51:02.879ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
60912025-08-26T21:51:02.879ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
60922025-08-26T21:51:02.880ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
60932025-08-26T21:51:02.880ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
60942025-08-26T21:51:02.880ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
60952025-08-26T21:51:02.880ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
60962025-08-26T21:51:02.880ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
60972025-08-26T21:51:02.880ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
60982025-08-26T21:51:02.896ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
60992025-08-26T21:51:02.897ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
61002025-08-26T21:51:02.898ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
61012025-08-26T21:51:02.902ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
61022025-08-26T21:51:02.903ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
61032025-08-26T21:51:02.905ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
61042025-08-26T21:51:02.906ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
61052025-08-26T21:51:02.906ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
61062025-08-26T21:51:02.907ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
61072025-08-26T21:51:02.907ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
61082025-08-26T21:51:02.907ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
61092025-08-26T21:51:02.907ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
61102025-08-26T21:51:02.907ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
61112025-08-26T21:51:02.914ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
61122025-08-26T21:51:02.914ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 8e720f02-dde8-4423-a6e3-91698373805a dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
61132025-08-26T21:51:02.914ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
61142025-08-26T21:51:02.914ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
61152025-08-26T21:51:02.916ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
61162025-08-26T21:51:02.916ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 55.024461ms iteration = 11 status = {"errors":[],"garbage_collect_requested":[]}
61172025-08-26T21:51:02.918ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
61182025-08-26T21:51:02.918ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
61192025-08-26T21:51:02.918ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
61202025-08-26T21:51:02.918ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
61212025-08-26T21:51:02.920ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
61222025-08-26T21:51:02.921ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
61232025-08-26T21:51:02.921ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
61242025-08-26T21:51:02.922ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
61252025-08-26T21:51:02.922ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
61262025-08-26T21:51:02.922ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
61272025-08-26T21:51:02.926ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
61282025-08-26T21:51:02.926ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
61292025-08-26T21:51:02.926ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
61302025-08-26T21:51:02.926ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
61312025-08-26T21:51:02.926ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
61322025-08-26T21:51:02.927ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-08-26T21:45:59.866217Z, time_modified: 2025-08-26T21:45:59.866217Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 9396aa55-de6f-4d6f-96db-d28e1c76fa9c, db_silo: Silo { identity: SiloIdentity { id: 9396aa55-de6f-4d6f-96db-d28e1c76fa9c, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-08-26T21:46:05.245521Z, time_modified: 2025-08-26T21:46:05.245521Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "e8529d720e4b0de099753dcdb373abb870af04baacd7c7dd850f5254858081af" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 9396aa55-de6f-4d6f-96db-d28e1c76fa9c, db_silo: Silo { identity: SiloIdentity { id: 9396aa55-de6f-4d6f-96db-d28e1c76fa9c, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-08-26T21:46:05.245521Z, time_modified: 2025-08-26T21:46:05.245521Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "e8529d720e4b0de099753dcdb373abb870af04baacd7c7dd850f5254858081af" }] }) }
61332025-08-26T21:51:02.927ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 92.435204ms iteration = 6 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"9396aa55-de6f-4d6f-96db-d28e1c76fa9c","tls_certs":["e8529d720e4b0de099753dcdb373abb870af04baacd7c7dd850f5254858081af"]}},"default_endpoint":{"silo_id":"9396aa55-de6f-4d6f-96db-d28e1c76fa9c","tls_certs":["e8529d720e4b0de099753dcdb373abb870af04baacd7c7dd850f5254858081af"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
61342025-08-26T21:51:02.927ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
61352025-08-26T21:51:02.927ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
61362025-08-26T21:51:02.927ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
61372025-08-26T21:51:02.927ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
61382025-08-26T21:51:02.927ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
61392025-08-26T21:51:02.927ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
61402025-08-26T21:51:02.927ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
61412025-08-26T21:51:02.928ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
61422025-08-26T21:51:02.928ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
61432025-08-26T21:51:02.928ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
61442025-08-26T21:51:02.928ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
61452025-08-26T21:51:02.928ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
61462025-08-26T21:51:02.928ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:170 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
61472025-08-26T21:51:02.928ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 67.475751ms iteration = 6 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
61482025-08-26T21:51:02.930ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
61492025-08-26T21:51:02.931ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
61502025-08-26T21:51:02.932ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
61512025-08-26T21:51:02.932ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
61522025-08-26T21:51:02.932ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 71.236456ms iteration = 31 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
61532025-08-26T21:51:02.932ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
61542025-08-26T21:51:02.933ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
61552025-08-26T21:51:02.933ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
61562025-08-26T21:51:02.934ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
61572025-08-26T21:51:02.935ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Inventory result = Ok(())
61582025-08-26T21:51:02.935ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
61592025-08-26T21:51:02.935ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
61602025-08-26T21:51:02.935ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
61612025-08-26T21:51:02.935ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 8e720f02-dde8-4423-a6e3-91698373805a dns_zone_name = control-plane.oxide.internal found_so_far = 59 version = 1
61622025-08-26T21:51:02.935ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
61632025-08-26T21:51:02.935ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2025-08-26 21:46:04.065217 UTC dns_group = internal generation = 1
61642025-08-26T21:51:02.935ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 101.350725ms iteration = 6 status = {"generation":1}
61652025-08-26T21:51:02.935ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
61662025-08-26T21:51:02.935ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
61672025-08-26T21:51:02.936ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
61682025-08-26T21:51:02.936ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
61692025-08-26T21:51:02.937ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
61702025-08-26T21:51:02.937ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
61712025-08-26T21:51:02.937ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
61722025-08-26T21:51:02.937ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
61732025-08-26T21:51:02.937ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
61742025-08-26T21:51:02.938ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 77.415255ms iteration = 11 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
61752025-08-26T21:51:02.939ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
61762025-08-26T21:51:02.939ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:138 found = 0 not_ready_to_be_deleted = 0
61772025-08-26T21:51:02.939ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 77.869596ms iteration = 6 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
61782025-08-26T21:51:02.939ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
61792025-08-26T21:51:02.940ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
61802025-08-26T21:51:02.940ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1166, tv_nsec: 798268070 } })
61812025-08-26T21:51:02.940ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1205 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
61822025-08-26T21:51:02.941ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 80.259299ms iteration = 11 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
61832025-08-26T21:51:02.944ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:550 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
61842025-08-26T21:51:02.944ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 83.435911ms iteration = 11 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
61852025-08-26T21:51:02.946ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
61862025-08-26T21:51:02.947ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
61872025-08-26T21:51:02.948ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
61882025-08-26T21:51:02.949ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
61892025-08-26T21:51:02.950ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
61902025-08-26T21:51:02.955ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: dd0c655a-95c2-48fc-b4c3-9733ed4b0faa, lookup_type: ById(dd0c655a-95c2-48fc-b4c3-9733ed4b0faa) } result = Ok(())
61912025-08-26T21:51:02.955ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
61922025-08-26T21:51:02.955ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
61932025-08-26T21:51:02.955ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
61942025-08-26T21:51:02.955ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 82.417858ms iteration = 31 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
61952025-08-26T21:51:02.957ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 96.055616ms iteration = 11 status = 0
61962025-08-26T21:51:02.957ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:407
61972025-08-26T21:51:02.957ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 96.422723ms iteration = 11 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
61982025-08-26T21:51:02.973ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
61992025-08-26T21:51:02.973ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
62002025-08-26T21:51:02.973ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
62012025-08-26T21:51:02.973ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
62022025-08-26T21:51:02.973ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
62032025-08-26T21:51:02.974ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
62042025-08-26T21:51:02.975ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
62052025-08-26T21:51:02.975ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
62062025-08-26T21:51:02.975ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
62072025-08-26T21:51:02.976ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
62082025-08-26T21:51:02.977ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62092025-08-26T21:51:02.981ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
62102025-08-26T21:51:02.981ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
62112025-08-26T21:51:02.981ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
62122025-08-26T21:51:02.983ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
62132025-08-26T21:51:02.984ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
62142025-08-26T21:51:02.985ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
62152025-08-26T21:51:02.985ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
62162025-08-26T21:51:02.986ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
62172025-08-26T21:51:02.986ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
62182025-08-26T21:51:02.987ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62192025-08-26T21:51:02.987ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
62202025-08-26T21:51:02.987ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
62212025-08-26T21:51:02.989ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
62222025-08-26T21:51:02.989ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
62232025-08-26T21:51:02.989ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
62242025-08-26T21:51:02.989ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
62252025-08-26T21:51:02.989ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
62262025-08-26T21:51:02.990ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62272025-08-26T21:51:02.990ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
62282025-08-26T21:51:02.990ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = dfb404ab-acb8-43de-afe8-b429af0c161d dns_zone_name = oxide.test found_so_far = 0 version = 2
62292025-08-26T21:51:02.990ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
62302025-08-26T21:51:02.990ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
62312025-08-26T21:51:02.991ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
62322025-08-26T21:51:02.991ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
62332025-08-26T21:51:02.991ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62342025-08-26T21:51:02.992ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62352025-08-26T21:51:02.992ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
62362025-08-26T21:51:02.992ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
62372025-08-26T21:51:02.993ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
62382025-08-26T21:51:02.993ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
62392025-08-26T21:51:02.993ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
62402025-08-26T21:51:02.993ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62412025-08-26T21:51:02.994ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
62422025-08-26T21:51:02.994ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
62432025-08-26T21:51:02.994ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
62442025-08-26T21:51:02.994ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
62452025-08-26T21:51:02.994ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
62462025-08-26T21:51:02.995ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62472025-08-26T21:51:02.998ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
62482025-08-26T21:51:02.998ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
62492025-08-26T21:51:02.998ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
62502025-08-26T21:51:02.998ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
62512025-08-26T21:51:02.998ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
62522025-08-26T21:51:02.998ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
62532025-08-26T21:51:02.998ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
62542025-08-26T21:51:02.998ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
62552025-08-26T21:51:02.998ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
62562025-08-26T21:51:02.999ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62572025-08-26T21:51:03.000ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
62582025-08-26T21:51:03.000ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
62592025-08-26T21:51:03.000ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = dfb404ab-acb8-43de-afe8-b429af0c161d dns_zone_name = oxide.test found_so_far = 4 version = 2
62602025-08-26T21:51:03.000ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
62612025-08-26T21:51:03.000ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2025-08-26 21:46:05.296205 UTC dns_group = external generation = 2
62622025-08-26T21:51:03.000ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 165.844719ms iteration = 6 status = {"generation":2}
62632025-08-26T21:51:03.001ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
62642025-08-26T21:51:03.001ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
62652025-08-26T21:51:03.001ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
62662025-08-26T21:51:03.001ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
62672025-08-26T21:51:03.001ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
62682025-08-26T21:51:03.004ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
62692025-08-26T21:51:03.004ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
62702025-08-26T21:51:03.006ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
62712025-08-26T21:51:03.006ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
62722025-08-26T21:51:03.009ZINFOnexus (BackgroundTasks): ensured all Crucible datasets present in inventory have database records background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/crucible_dataset.rs:113 num_already_exist = 9 num_inserted = 0 num_not_in_inventory = 0
62732025-08-26T21:51:03.011ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_rendezvous elapsed = 148.037237ms iteration = 3 status = {"blueprint_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","inventory_collection_id":"22ce40dd-e0b9-460a-8151-60c0f8e6fa4f","stats":{"crucible_dataset":{"num_already_exist":9,"num_inserted":0,"num_not_in_inventory":0},"debug_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0}}}
62742025-08-26T21:51:03.014ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 153.096931ms iteration = 11 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
62752025-08-26T21:51:03.019ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
62762025-08-26T21:51:03.020ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
62772025-08-26T21:51:03.025ZINFOnexus (BackgroundTasks): instance reincarnation completed successfully background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:113 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0
62782025-08-26T21:51:03.025ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 163.961276ms iteration = 6 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
62792025-08-26T21:51:03.027ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
62802025-08-26T21:51:03.027ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
62812025-08-26T21:51:03.027ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 166.33753ms iteration = 11 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
62822025-08-26T21:51:03.028ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1213 zone_address = fd00:1122:3344:101::2
62832025-08-26T21:51:03.028ZDEBGnexus (MgsClient): client request background_task = nat_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
62842025-08-26T21:51:03.029ZDEBGnexus (MgsClient): client response background_task = nat_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "8f064edf-658a-4a82-a7a0-b7de4bc1cdbc", "content-length": "26", "date": "Tue, 26 Aug 2025 21:51:03 GMT"} })
62852025-08-26T21:51:03.029ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1216 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
62862025-08-26T21:51:03.031ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
62872025-08-26T21:51:03.044ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
62882025-08-26T21:51:03.045ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
62892025-08-26T21:51:03.046ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
62902025-08-26T21:51:03.046ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
62912025-08-26T21:51:03.048ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62922025-08-26T21:51:03.052ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
62932025-08-26T21:51:03.052ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
62942025-08-26T21:51:03.052ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
62952025-08-26T21:51:03.053ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
62962025-08-26T21:51:03.053ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
62972025-08-26T21:51:03.055ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
62982025-08-26T21:51:03.055ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
62992025-08-26T21:51:03.057ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
63002025-08-26T21:51:03.061ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
63012025-08-26T21:51:03.061ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
63022025-08-26T21:51:03.061ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
63032025-08-26T21:51:03.065ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
63042025-08-26T21:51:03.065ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
63052025-08-26T21:51:03.067ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
63062025-08-26T21:51:03.067ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
63072025-08-26T21:51:03.069ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
63082025-08-26T21:51:03.069ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
63092025-08-26T21:51:03.072ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
63102025-08-26T21:51:03.072ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
63112025-08-26T21:51:03.073ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
63122025-08-26T21:51:03.074ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
63132025-08-26T21:51:03.075ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
63142025-08-26T21:51:03.075ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
63152025-08-26T21:51:03.076ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
63162025-08-26T21:51:03.077ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
63172025-08-26T21:51:03.078ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
63182025-08-26T21:51:03.081ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
63192025-08-26T21:51:03.081ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
63202025-08-26T21:51:03.082ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
63212025-08-26T21:51:03.083ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
63222025-08-26T21:51:03.083ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
63232025-08-26T21:51:03.084ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
63242025-08-26T21:51:03.085ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
63252025-08-26T21:51:03.086ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
63262025-08-26T21:51:03.090ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
63272025-08-26T21:51:03.090ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
63282025-08-26T21:51:03.090ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
63292025-08-26T21:51:03.096ZDEBGnexus (service-firewall-rule-progator): resolved names for firewall rules background_task = service_firewall_rule_propagation instance_interfaces = {} subnet_interfaces = {Name("nexus"): [NetworkInterface { id: 0802405a-a5f2-402c-94da-ccafb58bccb5, kind: Service { id: 4996dcc4-c530-4f64-b407-fa372462bd9e }, name: Name("nexus-4996dcc4-c530-4f64-b407-fa372462bd9e"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 186, 56])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 1acb6bfd-558b-4271-bbf5-58e0d089b710, kind: Service { id: 0205eaf0-3fc6-4ad5-b09b-732af233e9c1 }, name: Name("nexus-0205eaf0-3fc6-4ad5-b09b-732af233e9c1"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 243, 142])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: cbb99fa2-edc6-497d-9c47-8dd2642f8945, kind: Service { id: 5d0959e5-47a6-4bf3-a52b-57950b9d900d }, name: Name("nexus-5d0959e5-47a6-4bf3-a52b-57950b9d900d"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 162, 80])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], Name("external-dns"): [NetworkInterface { id: 0106180c-e6f8-4d65-969e-4fbbddf596a1, kind: Service { id: 60abf731-31e1-4111-a3e2-65da89eaed87 }, name: Name("external-dns-60abf731-31e1-4111-a3e2-65da89eaed87"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 156, 81])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: b69ae953-8f67-4cb3-9521-db9e707a6f2d, kind: Service { id: 84c849eb-0949-41d9-ad02-9a5509144564 }, name: Name("external-dns-84c849eb-0949-41d9-ad02-9a5509144564"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 240, 136])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }]} subnet_networks = {Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })], Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })]} vpc_interfaces = {}
63302025-08-26T21:51:03.096ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
63312025-08-26T21:51:03.096ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
63322025-08-26T21:51:03.098ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
63332025-08-26T21:51:03.100ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Fleet result = Ok(())
63342025-08-26T21:51:03.100ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
63352025-08-26T21:51:03.100ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
63362025-08-26T21:51:03.102ZDEBGnexus (service-firewall-rule-progator): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified. background_task = service_firewall_rule_propagation
63372025-08-26T21:51:03.102ZDEBGnexus (service-firewall-rule-progator): resolved firewall rules for sled agents background_task = service_firewall_rule_propagation sled_agent_rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 0106180c-e6f8-4d65-969e-4fbbddf596a1, kind: Service { id: 60abf731-31e1-4111-a3e2-65da89eaed87 }, name: Name("external-dns-60abf731-31e1-4111-a3e2-65da89eaed87"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 156, 81])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: b69ae953-8f67-4cb3-9521-db9e707a6f2d, kind: Service { id: 84c849eb-0949-41d9-ad02-9a5509144564 }, name: Name("external-dns-84c849eb-0949-41d9-ad02-9a5509144564"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 240, 136])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 0802405a-a5f2-402c-94da-ccafb58bccb5, kind: Service { id: 4996dcc4-c530-4f64-b407-fa372462bd9e }, name: Name("nexus-4996dcc4-c530-4f64-b407-fa372462bd9e"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 186, 56])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 1acb6bfd-558b-4271-bbf5-58e0d089b710, kind: Service { id: 0205eaf0-3fc6-4ad5-b09b-732af233e9c1 }, name: Name("nexus-0205eaf0-3fc6-4ad5-b09b-732af233e9c1"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 243, 142])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: cbb99fa2-edc6-497d-9c47-8dd2642f8945, kind: Service { id: 5d0959e5-47a6-4bf3-a52b-57950b9d900d }, name: Name("nexus-5d0959e5-47a6-4bf3-a52b-57950b9d900d"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 162, 80])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 0106180c-e6f8-4d65-969e-4fbbddf596a1, kind: Service { id: 60abf731-31e1-4111-a3e2-65da89eaed87 }, name: Name("external-dns-60abf731-31e1-4111-a3e2-65da89eaed87"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 156, 81])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: b69ae953-8f67-4cb3-9521-db9e707a6f2d, kind: Service { id: 84c849eb-0949-41d9-ad02-9a5509144564 }, name: Name("external-dns-84c849eb-0949-41d9-ad02-9a5509144564"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 240, 136])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: None, filter_protocols: Some([Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 3, code: Some(IcmpParamRange { first: 3, last: 4 }) })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 5, code: None })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 11, code: None }))]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 0802405a-a5f2-402c-94da-ccafb58bccb5, kind: Service { id: 4996dcc4-c530-4f64-b407-fa372462bd9e }, name: Name("nexus-4996dcc4-c530-4f64-b407-fa372462bd9e"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 186, 56])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 1acb6bfd-558b-4271-bbf5-58e0d089b710, kind: Service { id: 0205eaf0-3fc6-4ad5-b09b-732af233e9c1 }, name: Name("nexus-0205eaf0-3fc6-4ad5-b09b-732af233e9c1"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 243, 142])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: cbb99fa2-edc6-497d-9c47-8dd2642f8945, kind: Service { id: 5d0959e5-47a6-4bf3-a52b-57950b9d900d }, name: Name("nexus-5d0959e5-47a6-4bf3-a52b-57950b9d900d"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 162, 80])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }]
63382025-08-26T21:51:03.102ZDEBGnexus (service-firewall-rule-progator): resolved 3 rules for sleds background_task = service_firewall_rule_propagation
63392025-08-26T21:51:03.121ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 260.210846ms iteration = 12 status = {"physical_disks_added":0}
63402025-08-26T21:51:03.123ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1213 zone_address = fd00:1122:3344:101::2
63412025-08-26T21:51:03.123ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
63422025-08-26T21:51:03.124ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "f9597334-7023-472f-b67b-4a134aaa58b6", "content-length": "26", "date": "Tue, 26 Aug 2025 21:51:03 GMT"} })
63432025-08-26T21:51:03.124ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1216 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
63442025-08-26T21:51:03.125ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
63452025-08-26T21:51:03.168ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
63462025-08-26T21:51:03.168ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
63472025-08-26T21:51:03.169ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
63482025-08-26T21:51:03.170ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
63492025-08-26T21:51:03.171ZDEBGnexus (service-firewall-rule-progator): resolved sleds for vpc oxide-services background_task = service_firewall_rule_propagation vpc_to_sled = [Sled { identity: SledIdentity { id: dd0c655a-95c2-48fc-b4c3-9733ed4b0faa, time_created: 2025-08-26T21:46:02.113579Z, time_modified: 2025-08-26T21:46:02.113579Z }, time_deleted: None, rcgen: Generation(Generation(19)), rack_id: e37a46f2-f789-4ca4-9841-f8c6dd705964, is_scrimlet: true, serial_number: "sock", part_number: "i86pc", revision: SqlU32(0), usable_hardware_threads: SqlU32(48), usable_physical_ram: ByteCount(ByteCount(102952026112)), reservoir_size: ByteCount(ByteCount(56094621696)), ip: fd00:1122:3344:101::1, port: SqlU16(12345), last_used_address: fd00:1122:3344:101::ffff, policy: InService, state: Active, sled_agent_gen: Generation(Generation(1)), repo_depot_port: SqlU16(12348), cpu_family: AmdMilan }]
63502025-08-26T21:51:03.171ZDEBGnexus (service-firewall-rule-progator): sending firewall rules to sled agents background_task = service_firewall_rule_propagation
63512025-08-26T21:51:03.172ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Fleet result = Ok(())
63522025-08-26T21:51:03.172ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
63532025-08-26T21:51:03.172ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
63542025-08-26T21:51:03.174ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
63552025-08-26T21:51:03.174ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
63562025-08-26T21:51:03.176ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
63572025-08-26T21:51:03.178ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Fleet result = Ok(())
63582025-08-26T21:51:03.178ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
63592025-08-26T21:51:03.178ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
63602025-08-26T21:51:03.180ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = tuf_artifact_replication body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/artifacts-config
63612025-08-26T21:51:03.181ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = tuf_artifact_replication result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/artifacts-config", status: 204, headers: {"x-request-id": "0d169eca-6376-4589-8198-31a16b517481", "date": "Tue, 26 Aug 2025 21:51:03 GMT"} })
63622025-08-26T21:51:03.181ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = tuf_artifact_replication body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/artifacts
63632025-08-26T21:51:03.181ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = tuf_artifact_replication result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/artifacts", status: 200, headers: {"content-type": "application/json", "x-request-id": "da7b777f-9d9b-4543-8a14-cefe33e31299", "content-length": "26", "date": "Tue, 26 Aug 2025 21:51:02 GMT"} })
63642025-08-26T21:51:03.181ZINFOnexus (BackgroundTasks): Successfully got artifact list background_task = tuf_artifact_replication file = nexus/src/app/background/tasks/tuf_artifact_replication.rs:672 sled = http://[fd00:1122:3344:101::1]:12345
63652025-08-26T21:51:03.182ZDEBGnexus (BackgroundTasks): activation complete background_task = tuf_artifact_replication elapsed = 321.485347ms iteration = 2 status = {"generation":1,"last_run_counters":{"copy_err":0,"copy_ok":0,"list_err":0,"list_ok":1,"put_config_err":0,"put_config_ok":1,"put_err":0,"put_ok":0},"lifetime_counters":{"copy_err":0,"copy_ok":0,"list_err":0,"list_ok":2,"put_config_err":0,"put_config_ok":2,"put_err":0,"put_ok":0},"local_repos":0,"request_debug_ringbuf":[{"generation":1,"operation":"put_config","target_sled":"dd0c655a-95c2-48fc-b4c3-9733ed4b0faa","time":"2025-08-26T21:46:03.195562885Z"},{"operation":"list","target_sled":"dd0c655a-95c2-48fc-b4c3-9733ed4b0faa","time":"2025-08-26T21:46:03.196425124Z"},{"generation":1,"operation":"put_config","target_sled":"dd0c655a-95c2-48fc-b4c3-9733ed4b0faa","time":"2025-08-26T21:51:03.181001208Z"},{"operation":"list","target_sled":"dd0c655a-95c2-48fc-b4c3-9733ed4b0faa","time":"2025-08-26T21:51:03.181295604Z"}]}
63662025-08-26T21:51:03.245ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
63672025-08-26T21:51:03.269ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1213 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 zone_address = fd00:1122:3344:101::2
63682025-08-26T21:51:03.269ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
63692025-08-26T21:51:03.269ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "d12e4145-e8c6-4cae-8e6e-42cb09778318", "content-length": "26", "date": "Tue, 26 Aug 2025 21:51:03 GMT"} })
63702025-08-26T21:51:03.269ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1216 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
63712025-08-26T21:51:03.271ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
63722025-08-26T21:51:03.285ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
63732025-08-26T21:51:03.286ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "b3f2aa59-2ccb-4ecb-97ff-f3ae46755bb1", "content-length": "691", "date": "Tue, 26 Aug 2025 21:51:03 GMT"} })
63742025-08-26T21:51:03.286ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
63752025-08-26T21:51:03.286ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
63762025-08-26T21:51:03.286ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
63772025-08-26T21:51:03.287ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 426.7805ms iteration = 11 status = {}
63782025-08-26T21:51:03.318ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
63792025-08-26T21:51:03.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
63802025-08-26T21:51:03.320ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 485.860992ms iteration = 11 status = null
63812025-08-26T21:51:03.355ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 520.494123ms iteration = 11 status = {}
63822025-08-26T21:51:03.385ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
63832025-08-26T21:51:03.386ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
63842025-08-26T21:51:03.388ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
63852025-08-26T21:51:03.388ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
63862025-08-26T21:51:03.390ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
63872025-08-26T21:51:03.393ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Sled { parent: Fleet, key: dd0c655a-95c2-48fc-b4c3-9733ed4b0faa, lookup_type: ById(dd0c655a-95c2-48fc-b4c3-9733ed4b0faa) } result = Ok(())
63882025-08-26T21:51:03.397ZDEBGnexus (service-firewall-rule-progator): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = service_firewall_rule_propagation body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules
63892025-08-26T21:51:03.398ZDEBGnexus (service-firewall-rule-progator): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = service_firewall_rule_propagation result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules", status: 204, headers: {"x-request-id": "5fdbbcfe-d7ae-4627-b490-e787e6f8ba2b", "date": "Tue, 26 Aug 2025 21:51:03 GMT"} })
63902025-08-26T21:51:03.402ZINFOnexus (service-firewall-rule-progator): updated firewall rules on 1 sleds background_task = service_firewall_rule_propagation file = nexus/networking/src/firewall_rules.rs:491
63912025-08-26T21:51:03.403ZDEBGnexus (service-firewall-rule-progator): successfully propagated service firewall rules background_task = service_firewall_rule_propagation elapsed = 541.717512ms
63922025-08-26T21:51:03.403ZDEBGnexus (BackgroundTasks): activation complete background_task = service_firewall_rule_propagation elapsed = 541.725682ms iteration = 2 status = {}
63932025-08-26T21:51:03.462ZINFOnexus (BackgroundTasks): VPC route manager sled dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
63942025-08-26T21:51:03.462ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
63952025-08-26T21:51:03.462ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "b0576e3d-13eb-4f71-938a-1f7253f52098", "content-length": "372", "date": "Tue, 26 Aug 2025 21:51:03 GMT"} })
63962025-08-26T21:51:03.462ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
63972025-08-26T21:51:03.462ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
63982025-08-26T21:51:03.476ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
63992025-08-26T21:51:03.476ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
64002025-08-26T21:51:03.476ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "b8993d67-5feb-433a-a9b1-8713be8ccc36", "date": "Tue, 26 Aug 2025 21:51:03 GMT"} })
64012025-08-26T21:51:03.476ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
64022025-08-26T21:51:03.476ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
64032025-08-26T21:51:03.478ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
64042025-08-26T21:51:03.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
64052025-08-26T21:51:03.482ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
64062025-08-26T21:51:03.482ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
64072025-08-26T21:51:03.488ZINFOnexus (BackgroundTasks): VPC route manager sled dd0c655a-95c2-48fc-b4c3-9733ed4b0faa push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
64082025-08-26T21:51:03.488ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
64092025-08-26T21:51:03.489ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "ac457829-2d37-4906-85ef-8948eb15e054", "date": "Tue, 26 Aug 2025 21:51:03 GMT"} })
64102025-08-26T21:51:03.491ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 629.714615ms iteration = 11 status = {}
64112025-08-26T21:51:03.527ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
64122025-08-26T21:51:03.528ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "73d4b428-32e5-48a3-99ba-11125b67f2fd", "date": "Tue, 26 Aug 2025 21:51:03 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
64132025-08-26T21:51:03.528ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
64142025-08-26T21:51:03.530ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "3fef2104-2cbc-4615-9e67-3169d4d48d36", "date": "Tue, 26 Aug 2025 21:51:03 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
64152025-08-26T21:51:03.530ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
64162025-08-26T21:51:03.532ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
64172025-08-26T21:51:03.532ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 697.935323ms iteration = 8 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
64182025-08-26T21:51:03.537ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
64192025-08-26T21:51:03.593ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
64202025-08-26T21:51:03.594ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
64212025-08-26T21:51:03.596ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
64222025-08-26T21:51:03.596ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: d24c5c55-3a21-4a91-aeb4-45362e822df6, rack_id: e37a46f2-f789-4ca4-9841-f8c6dd705964, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(88d84f37-a4c8-4abc-af1f-27987b0d39d9) } rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0
64232025-08-26T21:51:03.596ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
64242025-08-26T21:51:03.596ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "44d35473-9445-49c1-a2d3-121718401d78", "date": "Tue, 26 Aug 2025 21:51:03 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
64252025-08-26T21:51:03.596ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
64262025-08-26T21:51:03.596ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
64272025-08-26T21:51:03.598ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "28ee74eb-586f-4b22-b0ac-00e82985e278", "date": "Tue, 26 Aug 2025 21:51:03 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
64282025-08-26T21:51:03.598ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
64292025-08-26T21:51:03.600ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "cbaa4379-7e0a-44fd-9efe-0d7d01e0702f", "date": "Tue, 26 Aug 2025 21:51:03 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
64302025-08-26T21:51:03.600ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
64312025-08-26T21:51:03.602ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
64322025-08-26T21:51:03.602ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 768.448582ms iteration = 8 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
64332025-08-26T21:51:03.608ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1832 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
64342025-08-26T21:51:03.608ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1859 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 88d84f37-a4c8-4abc-af1f-27987b0d39d9
64352025-08-26T21:51:03.609ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
64362025-08-26T21:51:03.609ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
64372025-08-26T21:51:03.609ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
64382025-08-26T21:51:03.609ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
64392025-08-26T21:51:03.609ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
64402025-08-26T21:51:03.609ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2006 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
64412025-08-26T21:51:03.609ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:408 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
64422025-08-26T21:51:03.609ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2042 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
64432025-08-26T21:51:03.609ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
64442025-08-26T21:51:03.609ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
64452025-08-26T21:51:03.609ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
64462025-08-26T21:51:03.612ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0
64472025-08-26T21:51:03.612ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
64482025-08-26T21:51:03.722ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "eeef2b16-b87f-4b64-91f8-f57432d83cb9", "date": "Tue, 26 Aug 2025 21:51:03 GMT"} })
64492025-08-26T21:51:03.722ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch = Switch0
64502025-08-26T21:51:03.722ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
64512025-08-26T21:51:03.722ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "0a0eb81a-25d7-4287-a4f5-cbf97b23c712", "content-length": "682", "date": "Tue, 26 Aug 2025 21:51:03 GMT"} })
64522025-08-26T21:51:03.722ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
64532025-08-26T21:51:03.722ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
64542025-08-26T21:51:03.726ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
64552025-08-26T21:51:03.726ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
64562025-08-26T21:51:03.730ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
64572025-08-26T21:51:03.731ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
64582025-08-26T21:51:03.733ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
64592025-08-26T21:51:03.733ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
64602025-08-26T21:51:03.741ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 879.741191ms iteration = 11 status = {}
64612025-08-26T21:51:05.353ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 290 local_addr = [fd00:1122:3344:101::c]:42016 method = GET remote_addr = [fd00:1122:3344:101::d]:37550 req_id = cc83abd7-85ca-41ac-b19d-cac918791200 response_code = 200 uri = /5d0959e5-47a6-4bf3-a52b-57950b9d900d
64622025-08-26T21:51:07.848ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 62 reason = Timeout
64632025-08-26T21:51:07.848ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
64642025-08-26T21:51:07.848ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
64652025-08-26T21:51:07.850ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
64662025-08-26T21:51:07.853ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
64672025-08-26T21:51:07.853ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
64682025-08-26T21:51:07.853ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
64692025-08-26T21:51:07.855ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
64702025-08-26T21:51:07.855ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.150512ms iteration = 62 status = {"chicken_switches_updated":false}
64712025-08-26T21:51:12.847ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 63 reason = Timeout
64722025-08-26T21:51:12.847ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
64732025-08-26T21:51:12.847ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
64742025-08-26T21:51:12.849ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
64752025-08-26T21:51:12.852ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
64762025-08-26T21:51:12.852ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
64772025-08-26T21:51:12.852ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
64782025-08-26T21:51:12.855ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
64792025-08-26T21:51:12.855ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.583501ms iteration = 63 status = {"chicken_switches_updated":false}
64802025-08-26T21:51:12.861ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 32 reason = Timeout
64812025-08-26T21:51:12.861ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
64822025-08-26T21:51:12.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
64832025-08-26T21:51:12.863ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
64842025-08-26T21:51:12.863ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
64852025-08-26T21:51:12.872ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 32 reason = Timeout
64862025-08-26T21:51:12.872ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
64872025-08-26T21:51:12.873ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
64882025-08-26T21:51:12.874ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
64892025-08-26T21:51:12.877ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
64902025-08-26T21:51:12.877ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
64912025-08-26T21:51:12.877ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
64922025-08-26T21:51:12.878ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
64932025-08-26T21:51:12.878ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
64942025-08-26T21:51:12.880ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
64952025-08-26T21:51:12.883ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
64962025-08-26T21:51:12.883ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
64972025-08-26T21:51:12.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
64982025-08-26T21:51:12.899ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
64992025-08-26T21:51:12.899ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 26.852965ms iteration = 32 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
65002025-08-26T21:51:12.899ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 38.825456ms iteration = 32 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
65012025-08-26T21:51:15.353ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 274 local_addr = [fd00:1122:3344:101::c]:42016 method = GET remote_addr = [fd00:1122:3344:101::d]:37550 req_id = 2a9dc540-561f-48ed-860a-f009a5eb46f8 response_code = 200 uri = /5d0959e5-47a6-4bf3-a52b-57950b9d900d
65022025-08-26T21:51:17.848ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 64 reason = Timeout
65032025-08-26T21:51:17.848ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
65042025-08-26T21:51:17.848ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
65052025-08-26T21:51:17.850ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
65062025-08-26T21:51:17.853ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
65072025-08-26T21:51:17.853ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
65082025-08-26T21:51:17.853ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
65092025-08-26T21:51:17.854ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
65102025-08-26T21:51:17.854ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 6.61688ms iteration = 64 status = {"chicken_switches_updated":false}
65112025-08-26T21:51:22.848ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 65 reason = Timeout
65122025-08-26T21:51:22.848ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
65132025-08-26T21:51:22.848ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
65142025-08-26T21:51:22.850ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
65152025-08-26T21:51:22.853ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
65162025-08-26T21:51:22.853ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
65172025-08-26T21:51:22.853ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
65182025-08-26T21:51:22.855ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
65192025-08-26T21:51:22.855ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.721814ms iteration = 65 status = {"chicken_switches_updated":false}
65202025-08-26T21:51:22.863ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 33 reason = Timeout
65212025-08-26T21:51:22.863ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
65222025-08-26T21:51:22.863ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
65232025-08-26T21:51:22.865ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
65242025-08-26T21:51:22.865ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
65252025-08-26T21:51:22.867ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.219195ms iteration = 33 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
65262025-08-26T21:51:22.872ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 33 reason = Timeout
65272025-08-26T21:51:22.872ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
65282025-08-26T21:51:22.873ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
65292025-08-26T21:51:22.875ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
65302025-08-26T21:51:22.878ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
65312025-08-26T21:51:22.878ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
65322025-08-26T21:51:22.878ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
65332025-08-26T21:51:22.880ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
65342025-08-26T21:51:22.880ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
65352025-08-26T21:51:22.881ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
65362025-08-26T21:51:22.884ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
65372025-08-26T21:51:22.884ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
65382025-08-26T21:51:22.884ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
65392025-08-26T21:51:22.897ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
65402025-08-26T21:51:22.897ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 24.268697ms iteration = 33 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
65412025-08-26T21:51:25.353ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 315 local_addr = [fd00:1122:3344:101::c]:42016 method = GET remote_addr = [fd00:1122:3344:101::d]:37550 req_id = 13d1af30-56cb-475a-a91f-3c2687be9cbe response_code = 200 uri = /5d0959e5-47a6-4bf3-a52b-57950b9d900d
65422025-08-26T21:51:27.848ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 66 reason = Timeout
65432025-08-26T21:51:27.848ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
65442025-08-26T21:51:27.848ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
65452025-08-26T21:51:27.850ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
65462025-08-26T21:51:27.853ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
65472025-08-26T21:51:27.853ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
65482025-08-26T21:51:27.853ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
65492025-08-26T21:51:27.855ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
65502025-08-26T21:51:27.855ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 6.894095ms iteration = 66 status = {"chicken_switches_updated":false}
65512025-08-26T21:51:32.833ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 12 reason = Timeout
65522025-08-26T21:51:32.833ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 12 reason = Timeout
65532025-08-26T21:51:32.833ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
65542025-08-26T21:51:32.833ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 12 reason = Timeout
65552025-08-26T21:51:32.833ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
65562025-08-26T21:51:32.834ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
65572025-08-26T21:51:32.834ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
65582025-08-26T21:51:32.836ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1196, tv_nsec: 694868419 } })
65592025-08-26T21:51:32.836ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1196, tv_nsec: 694873849 } })
65602025-08-26T21:51:32.837ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1205
65612025-08-26T21:51:32.837ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.756545ms iteration = 12 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
65622025-08-26T21:51:32.837ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1205
65632025-08-26T21:51:32.847ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 67 reason = Timeout
65642025-08-26T21:51:32.847ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
65652025-08-26T21:51:32.848ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
65662025-08-26T21:51:32.850ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
65672025-08-26T21:51:32.853ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
65682025-08-26T21:51:32.853ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
65692025-08-26T21:51:32.853ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
65702025-08-26T21:51:32.855ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
65712025-08-26T21:51:32.855ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.507139ms iteration = 67 status = {"chicken_switches_updated":false}
65722025-08-26T21:51:32.860ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 12 reason = Timeout
65732025-08-26T21:51:32.860ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
65742025-08-26T21:51:32.860ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 13 reason = Timeout
65752025-08-26T21:51:32.860ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 12 reason = Timeout
65762025-08-26T21:51:32.860ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 12 reason = Timeout
65772025-08-26T21:51:32.860ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
65782025-08-26T21:51:32.860ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
65792025-08-26T21:51:32.860ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
65802025-08-26T21:51:32.860ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 12 reason = Timeout
65812025-08-26T21:51:32.860ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
65822025-08-26T21:51:32.860ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 12 reason = Timeout
65832025-08-26T21:51:32.860ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
65842025-08-26T21:51:32.860ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
65852025-08-26T21:51:32.860ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 34 reason = Timeout
65862025-08-26T21:51:32.860ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
65872025-08-26T21:51:32.860ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 12 reason = Timeout
65882025-08-26T21:51:32.860ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
65892025-08-26T21:51:32.860ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 12 reason = Timeout
65902025-08-26T21:51:32.860ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 12 reason = Timeout
65912025-08-26T21:51:32.860ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
65922025-08-26T21:51:32.860ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
65932025-08-26T21:51:32.860ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
65942025-08-26T21:51:32.860ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
65952025-08-26T21:51:32.860ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
65962025-08-26T21:51:32.860ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
65972025-08-26T21:51:32.860ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
65982025-08-26T21:51:32.860ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
65992025-08-26T21:51:32.860ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
66002025-08-26T21:51:32.860ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
66012025-08-26T21:51:32.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
66022025-08-26T21:51:32.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
66032025-08-26T21:51:32.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
66042025-08-26T21:51:32.861ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 12 reason = Timeout
66052025-08-26T21:51:32.861ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 12 reason = Timeout
66062025-08-26T21:51:32.861ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
66072025-08-26T21:51:32.861ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 13.661\u{b5}s iteration = 12 status = {"disabled":true,"errors":[],"sps":[]}
66082025-08-26T21:51:32.861ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 12 reason = Timeout
66092025-08-26T21:51:32.861ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 12 reason = Timeout
66102025-08-26T21:51:32.861ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
66112025-08-26T21:51:32.861ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
66122025-08-26T21:51:32.861ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 12 reason = Timeout
66132025-08-26T21:51:32.861ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
66142025-08-26T21:51:32.861ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 12 reason = Timeout
66152025-08-26T21:51:32.861ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
66162025-08-26T21:51:32.861ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
66172025-08-26T21:51:32.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
66182025-08-26T21:51:32.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
66192025-08-26T21:51:32.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
66202025-08-26T21:51:32.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
66212025-08-26T21:51:32.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
66222025-08-26T21:51:32.862ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
66232025-08-26T21:51:32.862ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
66242025-08-26T21:51:32.862ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
66252025-08-26T21:51:32.863ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
66262025-08-26T21:51:32.863ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
66272025-08-26T21:51:32.863ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
66282025-08-26T21:51:32.863ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
66292025-08-26T21:51:32.864ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
66302025-08-26T21:51:32.865ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
66312025-08-26T21:51:32.865ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
66322025-08-26T21:51:32.866ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
66332025-08-26T21:51:32.866ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
66342025-08-26T21:51:32.867ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
66352025-08-26T21:51:32.867ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
66362025-08-26T21:51:32.867ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
66372025-08-26T21:51:32.867ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
66382025-08-26T21:51:32.867ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
66392025-08-26T21:51:32.867ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
66402025-08-26T21:51:32.867ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
66412025-08-26T21:51:32.867ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
66422025-08-26T21:51:32.867ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
66432025-08-26T21:51:32.867ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
66442025-08-26T21:51:32.867ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
66452025-08-26T21:51:32.867ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
66462025-08-26T21:51:32.867ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
66472025-08-26T21:51:32.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
66482025-08-26T21:51:32.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
66492025-08-26T21:51:32.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
66502025-08-26T21:51:32.868ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
66512025-08-26T21:51:32.868ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
66522025-08-26T21:51:32.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
66532025-08-26T21:51:32.870ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
66542025-08-26T21:51:32.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
66552025-08-26T21:51:32.870ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
66562025-08-26T21:51:32.870ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1196, tv_nsec: 728749013 } })
66572025-08-26T21:51:32.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
66582025-08-26T21:51:32.871ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1205 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
66592025-08-26T21:51:32.872ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 34 reason = Timeout
66602025-08-26T21:51:32.872ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
66612025-08-26T21:51:32.872ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
66622025-08-26T21:51:32.872ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
66632025-08-26T21:51:32.874ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
66642025-08-26T21:51:32.874ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
66652025-08-26T21:51:32.874ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
66662025-08-26T21:51:32.874ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
66672025-08-26T21:51:32.877ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
66682025-08-26T21:51:32.877ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
66692025-08-26T21:51:32.877ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
66702025-08-26T21:51:32.878ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
66712025-08-26T21:51:32.880ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
66722025-08-26T21:51:32.911ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1213 zone_address = fd00:1122:3344:101::2
66732025-08-26T21:51:32.911ZDEBGnexus (MgsClient): client request background_task = nat_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
66742025-08-26T21:51:32.911ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
66752025-08-26T21:51:32.911ZDEBGnexus (MgsClient): client response background_task = nat_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "9623e5e0-c7e7-44fe-9480-f956eee04896", "content-length": "26", "date": "Tue, 26 Aug 2025 21:51:32 GMT"} })
66762025-08-26T21:51:32.911ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1216 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
66772025-08-26T21:51:32.912ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
66782025-08-26T21:51:32.913ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
66792025-08-26T21:51:32.916ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
66802025-08-26T21:51:32.917ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
66812025-08-26T21:51:32.919ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 57.243271ms iteration = 12 status = {"errors":[],"requests_created_ok":[]}
66822025-08-26T21:51:32.919ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
66832025-08-26T21:51:32.920ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
66842025-08-26T21:51:32.920ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
66852025-08-26T21:51:32.920ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 58.749304ms iteration = 12 status = {"errors":[],"garbage_collect_requested":[]}
66862025-08-26T21:51:32.921ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
66872025-08-26T21:51:32.922ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
66882025-08-26T21:51:32.922ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 61.468636ms iteration = 12 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
66892025-08-26T21:51:32.922ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
66902025-08-26T21:51:32.922ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
66912025-08-26T21:51:32.923ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 61.263571ms iteration = 12 status = {"errors":[],"finish_invoked_ok":[]}
66922025-08-26T21:51:32.923ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
66932025-08-26T21:51:32.926ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
66942025-08-26T21:51:32.927ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1213 zone_address = fd00:1122:3344:101::2
66952025-08-26T21:51:32.927ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
66962025-08-26T21:51:32.927ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 67.292697ms iteration = 12 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
66972025-08-26T21:51:32.928ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "7754026a-556e-461e-8176-ac23550f58f7", "content-length": "26", "date": "Tue, 26 Aug 2025 21:51:32 GMT"} })
66982025-08-26T21:51:32.928ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1216 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
66992025-08-26T21:51:32.928ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
67002025-08-26T21:51:32.928ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
67012025-08-26T21:51:32.928ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
67022025-08-26T21:51:32.929ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
67032025-08-26T21:51:32.930ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:407
67042025-08-26T21:51:32.930ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 69.886575ms iteration = 12 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
67052025-08-26T21:51:32.930ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 68.816561ms iteration = 12 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
67062025-08-26T21:51:32.932ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:550 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
67072025-08-26T21:51:32.932ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 71.640825ms iteration = 12 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
67082025-08-26T21:51:32.936ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
67092025-08-26T21:51:32.936ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
67102025-08-26T21:51:32.937ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
67112025-08-26T21:51:32.938ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
67122025-08-26T21:51:32.938ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
67132025-08-26T21:51:32.940ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
67142025-08-26T21:51:32.940ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
67152025-08-26T21:51:32.940ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
67162025-08-26T21:51:32.940ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
67172025-08-26T21:51:32.942ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: dd0c655a-95c2-48fc-b4c3-9733ed4b0faa, lookup_type: ById(dd0c655a-95c2-48fc-b4c3-9733ed4b0faa) } result = Ok(())
67182025-08-26T21:51:32.942ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
67192025-08-26T21:51:32.943ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
67202025-08-26T21:51:32.945ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 84.830822ms iteration = 12 status = 0
67212025-08-26T21:51:32.957ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
67222025-08-26T21:51:32.957ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 85.16627ms iteration = 34 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
67232025-08-26T21:51:32.962ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
67242025-08-26T21:51:32.963ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
67252025-08-26T21:51:32.968ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 108.135868ms iteration = 34 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
67262025-08-26T21:51:32.968ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1213 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 zone_address = fd00:1122:3344:101::2
67272025-08-26T21:51:32.968ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
67282025-08-26T21:51:32.969ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "8435a970-d5ec-484e-bc40-762e47f37220", "content-length": "26", "date": "Tue, 26 Aug 2025 21:51:32 GMT"} })
67292025-08-26T21:51:32.969ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1216 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
67302025-08-26T21:51:32.970ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
67312025-08-26T21:51:32.971ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
67322025-08-26T21:51:32.973ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
67332025-08-26T21:51:32.973ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
67342025-08-26T21:51:32.973ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
67352025-08-26T21:51:33.004ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 143.023274ms iteration = 12 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
67362025-08-26T21:51:33.015ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
67372025-08-26T21:51:33.016ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
67382025-08-26T21:51:33.025ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 164.913038ms iteration = 13 status = {"physical_disks_added":0}
67392025-08-26T21:51:33.107ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
67402025-08-26T21:51:33.108ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
67412025-08-26T21:51:33.110ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 276.590615ms iteration = 12 status = null
67422025-08-26T21:51:33.122ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 288.631747ms iteration = 12 status = {}
67432025-08-26T21:51:33.132ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
67442025-08-26T21:51:33.133ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
67452025-08-26T21:51:33.134ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
67462025-08-26T21:51:33.137ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
67472025-08-26T21:51:33.137ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
67482025-08-26T21:51:33.137ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
67492025-08-26T21:51:33.139ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
67502025-08-26T21:51:33.141ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
67512025-08-26T21:51:33.141ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
67522025-08-26T21:51:33.141ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
67532025-08-26T21:51:33.149ZINFOnexus (BackgroundTasks): VPC route manager sled dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
67542025-08-26T21:51:33.149ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
67552025-08-26T21:51:33.150ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "247d4323-8909-472f-8ce0-2ace49844d67", "content-length": "372", "date": "Tue, 26 Aug 2025 21:51:32 GMT"} })
67562025-08-26T21:51:33.150ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
67572025-08-26T21:51:33.150ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
67582025-08-26T21:51:33.164ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
67592025-08-26T21:51:33.164ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
67602025-08-26T21:51:33.165ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "4140fb1c-9322-421c-a23a-a325c4555fc6", "date": "Tue, 26 Aug 2025 21:51:33 GMT"} })
67612025-08-26T21:51:33.165ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
67622025-08-26T21:51:33.165ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
67632025-08-26T21:51:33.167ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
67642025-08-26T21:51:33.167ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
67652025-08-26T21:51:33.169ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
67662025-08-26T21:51:33.170ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
67672025-08-26T21:51:33.172ZINFOnexus (BackgroundTasks): VPC route manager sled dd0c655a-95c2-48fc-b4c3-9733ed4b0faa push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
67682025-08-26T21:51:33.172ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
67692025-08-26T21:51:33.173ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "9ded1f5a-679f-4241-a5f2-d49ad3ed50e1", "date": "Tue, 26 Aug 2025 21:51:32 GMT"} })
67702025-08-26T21:51:33.174ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 314.473009ms iteration = 12 status = {}
67712025-08-26T21:51:33.206ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
67722025-08-26T21:51:33.207ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "4fa4a5a4-5500-4d1c-aef4-a747966287b6", "content-length": "691", "date": "Tue, 26 Aug 2025 21:51:33 GMT"} })
67732025-08-26T21:51:33.207ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
67742025-08-26T21:51:33.207ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
67752025-08-26T21:51:33.207ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
67762025-08-26T21:51:33.208ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 348.377384ms iteration = 12 status = {}
67772025-08-26T21:51:33.216ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
67782025-08-26T21:51:33.216ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
67792025-08-26T21:51:33.218ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: d24c5c55-3a21-4a91-aeb4-45362e822df6, rack_id: e37a46f2-f789-4ca4-9841-f8c6dd705964, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(88d84f37-a4c8-4abc-af1f-27987b0d39d9) } rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0
67802025-08-26T21:51:33.218ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
67812025-08-26T21:51:33.219ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
67822025-08-26T21:51:33.231ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1832 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
67832025-08-26T21:51:33.231ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1859 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 88d84f37-a4c8-4abc-af1f-27987b0d39d9
67842025-08-26T21:51:33.231ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
67852025-08-26T21:51:33.231ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
67862025-08-26T21:51:33.231ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
67872025-08-26T21:51:33.231ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
67882025-08-26T21:51:33.231ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
67892025-08-26T21:51:33.231ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2006 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
67902025-08-26T21:51:33.231ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:408 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
67912025-08-26T21:51:33.231ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2042 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
67922025-08-26T21:51:33.231ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
67932025-08-26T21:51:33.231ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
67942025-08-26T21:51:33.232ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
67952025-08-26T21:51:33.234ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0
67962025-08-26T21:51:33.234ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
67972025-08-26T21:51:33.343ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "4bcb4b0c-7d25-4a28-816a-b77d51c743b1", "date": "Tue, 26 Aug 2025 21:51:33 GMT"} })
67982025-08-26T21:51:33.343ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch = Switch0
67992025-08-26T21:51:33.343ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
68002025-08-26T21:51:33.343ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "904a7a7b-21bb-4ea5-ad8c-8223f3eceae5", "content-length": "682", "date": "Tue, 26 Aug 2025 21:51:33 GMT"} })
68012025-08-26T21:51:33.343ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
68022025-08-26T21:51:33.343ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
68032025-08-26T21:51:33.346ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
68042025-08-26T21:51:33.346ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
68052025-08-26T21:51:33.350ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
68062025-08-26T21:51:33.350ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
68072025-08-26T21:51:33.352ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
68082025-08-26T21:51:33.352ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
68092025-08-26T21:51:33.360ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 499.798456ms iteration = 12 status = {}
68102025-08-26T21:51:35.352ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 321 local_addr = [fd00:1122:3344:101::c]:42016 method = GET remote_addr = [fd00:1122:3344:101::d]:37550 req_id = 9d8cf588-fe6f-4cba-9734-77e4e474e0fe response_code = 200 uri = /5d0959e5-47a6-4bf3-a52b-57950b9d900d
68112025-08-26T21:51:37.847ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 68 reason = Timeout
68122025-08-26T21:51:37.848ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
68132025-08-26T21:51:37.848ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
68142025-08-26T21:51:37.850ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
68152025-08-26T21:51:37.853ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
68162025-08-26T21:51:37.853ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
68172025-08-26T21:51:37.853ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
68182025-08-26T21:51:37.855ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
68192025-08-26T21:51:37.855ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.720864ms iteration = 68 status = {"chicken_switches_updated":false}
68202025-08-26T21:51:42.847ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 69 reason = Timeout
68212025-08-26T21:51:42.847ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
68222025-08-26T21:51:42.848ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
68232025-08-26T21:51:42.850ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
68242025-08-26T21:51:42.852ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
68252025-08-26T21:51:42.852ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
68262025-08-26T21:51:42.852ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
68272025-08-26T21:51:42.854ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
68282025-08-26T21:51:42.854ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 6.980497ms iteration = 69 status = {"chicken_switches_updated":false}
68292025-08-26T21:51:42.860ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 35 reason = Timeout
68302025-08-26T21:51:42.860ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
68312025-08-26T21:51:42.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
68322025-08-26T21:51:42.872ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 35 reason = Timeout
68332025-08-26T21:51:42.872ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
68342025-08-26T21:51:42.873ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
68352025-08-26T21:51:42.874ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
68362025-08-26T21:51:42.877ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
68372025-08-26T21:51:42.877ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
68382025-08-26T21:51:42.877ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
68392025-08-26T21:51:42.879ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
68402025-08-26T21:51:42.880ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
68412025-08-26T21:51:42.881ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
68422025-08-26T21:51:42.884ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
68432025-08-26T21:51:42.884ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
68442025-08-26T21:51:42.884ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
68452025-08-26T21:51:42.896ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
68462025-08-26T21:51:42.896ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
68472025-08-26T21:51:42.900ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
68482025-08-26T21:51:42.900ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 27.624893ms iteration = 35 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
68492025-08-26T21:51:42.932ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 71.263016ms iteration = 35 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
68502025-08-26T21:51:45.352ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 320 local_addr = [fd00:1122:3344:101::c]:42016 method = GET remote_addr = [fd00:1122:3344:101::d]:37550 req_id = 2f0d2bee-2c91-426a-bf5b-46812d788d6e response_code = 200 uri = /5d0959e5-47a6-4bf3-a52b-57950b9d900d
68512025-08-26T21:51:47.847ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 70 reason = Timeout
68522025-08-26T21:51:47.847ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
68532025-08-26T21:51:47.848ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
68542025-08-26T21:51:47.849ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
68552025-08-26T21:51:47.852ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
68562025-08-26T21:51:47.852ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
68572025-08-26T21:51:47.852ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
68582025-08-26T21:51:47.854ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
68592025-08-26T21:51:47.854ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.027738ms iteration = 70 status = {"chicken_switches_updated":false}
68602025-08-26T21:51:52.848ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 71 reason = Timeout
68612025-08-26T21:51:52.848ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
68622025-08-26T21:51:52.848ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
68632025-08-26T21:51:52.850ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
68642025-08-26T21:51:52.853ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
68652025-08-26T21:51:52.853ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
68662025-08-26T21:51:52.853ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
68672025-08-26T21:51:52.855ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
68682025-08-26T21:51:52.855ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.193362ms iteration = 71 status = {"chicken_switches_updated":false}
68692025-08-26T21:51:52.860ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 36 reason = Timeout
68702025-08-26T21:51:52.860ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
68712025-08-26T21:51:52.860ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
68722025-08-26T21:51:52.862ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
68732025-08-26T21:51:52.863ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
68742025-08-26T21:51:52.864ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.391439ms iteration = 36 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
68752025-08-26T21:51:52.872ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 36 reason = Timeout
68762025-08-26T21:51:52.872ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
68772025-08-26T21:51:52.872ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
68782025-08-26T21:51:52.874ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
68792025-08-26T21:51:52.877ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
68802025-08-26T21:51:52.877ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
68812025-08-26T21:51:52.878ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
68822025-08-26T21:51:52.880ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
68832025-08-26T21:51:52.880ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
68842025-08-26T21:51:52.882ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
68852025-08-26T21:51:52.885ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
68862025-08-26T21:51:52.885ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
68872025-08-26T21:51:52.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
68882025-08-26T21:51:52.903ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
68892025-08-26T21:51:52.903ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 30.929947ms iteration = 36 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
68902025-08-26T21:51:55.353ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 313 local_addr = [fd00:1122:3344:101::c]:42016 method = GET remote_addr = [fd00:1122:3344:101::d]:37550 req_id = 4e689e19-bf97-4e6a-b713-31deb1cfd17e response_code = 200 uri = /5d0959e5-47a6-4bf3-a52b-57950b9d900d
68912025-08-26T21:51:57.848ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 72 reason = Timeout
68922025-08-26T21:51:57.848ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
68932025-08-26T21:51:57.848ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
68942025-08-26T21:51:57.850ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
68952025-08-26T21:51:57.853ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
68962025-08-26T21:51:57.853ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
68972025-08-26T21:51:57.853ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
68982025-08-26T21:51:57.855ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
68992025-08-26T21:51:57.855ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.322555ms iteration = 72 status = {"chicken_switches_updated":false}
69002025-08-26T21:52:02.833ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 7 reason = Timeout
69012025-08-26T21:52:02.833ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 7 reason = Timeout
69022025-08-26T21:52:02.833ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
69032025-08-26T21:52:02.833ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
69042025-08-26T21:52:02.833ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 13 reason = Timeout
69052025-08-26T21:52:02.833ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 7 reason = Timeout
69062025-08-26T21:52:02.833ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-08-26 21:42:02.833770494 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
69072025-08-26T21:52:02.833ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
69082025-08-26T21:52:02.833ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 7 reason = Timeout
69092025-08-26T21:52:02.833ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 7 reason = Timeout
69102025-08-26T21:52:02.833ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 13 reason = Timeout
69112025-08-26T21:52:02.833ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
69122025-08-26T21:52:02.833ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
69132025-08-26T21:52:02.833ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
69142025-08-26T21:52:02.833ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
69152025-08-26T21:52:02.833ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 9 reason = Timeout
69162025-08-26T21:52:02.833ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 7 reason = Timeout
69172025-08-26T21:52:02.833ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 9 reason = Timeout
69182025-08-26T21:52:02.833ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 13 reason = Timeout
69192025-08-26T21:52:02.833ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
69202025-08-26T21:52:02.834ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
69212025-08-26T21:52:02.834ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
69222025-08-26T21:52:02.834ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
69232025-08-26T21:52:02.834ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
69242025-08-26T21:52:02.834ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
69252025-08-26T21:52:02.834ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
69262025-08-26T21:52:02.834ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("60abf731-31e1-4111-a3e2-65da89eaed87.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("84c849eb-0949-41d9-ad02-9a5509144564.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("84c849eb-0949-41d9-ad02-9a5509144564.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::8)) }], valid_until: Instant { tv_sec: 1226, tv_nsec: 692476525 } })
69272025-08-26T21:52:02.834ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("852fd1a3-de9a-47e8-a667-5e8dab880ad3.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("c68f1501-294c-4874-92ca-8fc0546a5870.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("d741a72f-8745-4723-9e80-e1e72987266b.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("d741a72f-8745-4723-9e80-e1e72987266b.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:3::1)) }], valid_until: Instant { tv_sec: 1226, tv_nsec: 692479455 } })
69282025-08-26T21:52:02.834ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } dns_group = internal
69292025-08-26T21:52:02.835ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
69302025-08-26T21:52:02.835ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.129005ms iteration = 7 status = {"addresses":["[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:3::1]:5353"]}
69312025-08-26T21:52:02.835ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.258818ms iteration = 7 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
69322025-08-26T21:52:02.835ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
69332025-08-26T21:52:02.835ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
69342025-08-26T21:52:02.835ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
69352025-08-26T21:52:02.835ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
69362025-08-26T21:52:02.836ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
69372025-08-26T21:52:02.836ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 8 reason = Timeout
69382025-08-26T21:52:02.836ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:91 reason = blueprint disabled target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8
69392025-08-26T21:52:02.836ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 11.53\u{b5}s iteration = 8 status = {"enabled":false,"target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8"}
69402025-08-26T21:52:02.836ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1226, tv_nsec: 694783927 } })
69412025-08-26T21:52:02.836ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
69422025-08-26T21:52:02.836ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-08-26 21:42:02.833770494 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
69432025-08-26T21:52:02.836ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1205
69442025-08-26T21:52:02.836ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 3.209202ms iteration = 7 status = {"expiration":"2025-08-26T21:42:02.833770494Z","pruned":[]}
69452025-08-26T21:52:02.837ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.366046ms iteration = 13 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
69462025-08-26T21:52:02.838ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
69472025-08-26T21:52:02.838ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
69482025-08-26T21:52:02.838ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1226, tv_nsec: 696859674 } })
69492025-08-26T21:52:02.838ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
69502025-08-26T21:52:02.839ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1205
69512025-08-26T21:52:02.840ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
69522025-08-26T21:52:02.840ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
69532025-08-26T21:52:02.840ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
69542025-08-26T21:52:02.841ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
69552025-08-26T21:52:02.846ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
69562025-08-26T21:52:02.846ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
69572025-08-26T21:52:02.846ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
69582025-08-26T21:52:02.848ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 73 reason = Timeout
69592025-08-26T21:52:02.848ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
69602025-08-26T21:52:02.848ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
69612025-08-26T21:52:02.849ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
69622025-08-26T21:52:02.852ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
69632025-08-26T21:52:02.852ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
69642025-08-26T21:52:02.852ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
69652025-08-26T21:52:02.853ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
69662025-08-26T21:52:02.853ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 5.883332ms iteration = 73 status = {"chicken_switches_updated":false}
69672025-08-26T21:52:02.860ZDEBGnexus (BackgroundTasks): activating background_task = crdb_node_id_collector iteration = 4 reason = Timeout
69682025-08-26T21:52:02.860ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 13 reason = Timeout
69692025-08-26T21:52:02.860ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
69702025-08-26T21:52:02.860ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
69712025-08-26T21:52:02.861ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 7 reason = Timeout
69722025-08-26T21:52:02.861ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:53
69732025-08-26T21:52:02.861ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 13 reason = Timeout
69742025-08-26T21:52:02.861ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
69752025-08-26T21:52:02.861ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
69762025-08-26T21:52:02.861ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
69772025-08-26T21:52:02.861ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 9 reason = Timeout
69782025-08-26T21:52:02.861ZDEBGnexus (BackgroundTasks): blueprint planning disabled, doing nothing background_task = blueprint_planner
69792025-08-26T21:52:02.861ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 13 reason = Timeout
69802025-08-26T21:52:02.861ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 10.11\u{b5}s iteration = 9 status = "Disabled"
69812025-08-26T21:52:02.861ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
69822025-08-26T21:52:02.861ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
69832025-08-26T21:52:02.861ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 7 reason = Timeout
69842025-08-26T21:52:02.861ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 13 reason = Timeout
69852025-08-26T21:52:02.861ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
69862025-08-26T21:52:02.861ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
69872025-08-26T21:52:02.861ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 13 reason = Timeout
69882025-08-26T21:52:02.861ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
69892025-08-26T21:52:02.861ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 14 reason = Timeout
69902025-08-26T21:52:02.861ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
69912025-08-26T21:52:02.861ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 13 reason = Timeout
69922025-08-26T21:52:02.861ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
69932025-08-26T21:52:02.861ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 37 reason = Timeout
69942025-08-26T21:52:02.861ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
69952025-08-26T21:52:02.861ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 7 reason = Timeout
69962025-08-26T21:52:02.861ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
69972025-08-26T21:52:02.861ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 13 reason = Timeout
69982025-08-26T21:52:02.861ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
69992025-08-26T21:52:02.861ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 13 reason = Timeout
70002025-08-26T21:52:02.861ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
70012025-08-26T21:52:02.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
70022025-08-26T21:52:02.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
70032025-08-26T21:52:02.861ZDEBGnexus (BackgroundTasks): activating background_task = webhook_deliverator iteration = 7 reason = Timeout
70042025-08-26T21:52:02.861ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
70052025-08-26T21:52:02.861ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 13 reason = Timeout
70062025-08-26T21:52:02.861ZDEBGnexus (BackgroundTasks): roles background_task = webhook_deliverator roles = RoleSet { roles: {} }
70072025-08-26T21:52:02.861ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
70082025-08-26T21:52:02.861ZDEBGnexus (BackgroundTasks): activating background_task = alert_dispatcher iteration = 7 reason = Timeout
70092025-08-26T21:52:02.861ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
70102025-08-26T21:52:02.861ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 7 reason = Timeout
70112025-08-26T21:52:02.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
70122025-08-26T21:52:02.861ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
70132025-08-26T21:52:02.861ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 13 reason = Timeout
70142025-08-26T21:52:02.861ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 7.5\u{b5}s iteration = 13 status = {"disabled":true,"errors":[],"sps":[]}
70152025-08-26T21:52:02.861ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 13 reason = Timeout
70162025-08-26T21:52:02.861ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 13 reason = Timeout
70172025-08-26T21:52:02.861ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
70182025-08-26T21:52:02.861ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
70192025-08-26T21:52:02.861ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 13 reason = Timeout
70202025-08-26T21:52:02.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
70212025-08-26T21:52:02.861ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
70222025-08-26T21:52:02.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
70232025-08-26T21:52:02.861ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 13 reason = Timeout
70242025-08-26T21:52:02.861ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
70252025-08-26T21:52:02.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
70262025-08-26T21:52:02.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
70272025-08-26T21:52:02.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
70282025-08-26T21:52:02.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
70292025-08-26T21:52:02.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
70302025-08-26T21:52:02.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
70312025-08-26T21:52:02.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
70322025-08-26T21:52:02.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
70332025-08-26T21:52:02.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
70342025-08-26T21:52:02.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
70352025-08-26T21:52:02.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
70362025-08-26T21:52:02.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
70372025-08-26T21:52:02.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = webhook_deliverator resource = Database result = Ok(())
70382025-08-26T21:52:02.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
70392025-08-26T21:52:02.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = alert_dispatcher resource = Database result = Ok(())
70402025-08-26T21:52:02.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
70412025-08-26T21:52:02.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
70422025-08-26T21:52:02.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
70432025-08-26T21:52:02.862ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
70442025-08-26T21:52:02.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
70452025-08-26T21:52:02.862ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
70462025-08-26T21:52:02.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
70472025-08-26T21:52:02.862ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
70482025-08-26T21:52:02.862ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
70492025-08-26T21:52:02.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
70502025-08-26T21:52:02.862ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
70512025-08-26T21:52:02.863ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
70522025-08-26T21:52:02.863ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
70532025-08-26T21:52:02.863ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
70542025-08-26T21:52:02.864ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
70552025-08-26T21:52:02.864ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 3.416448ms iteration = 7 status = {"errors":[],"found_port_ok":[]}
70562025-08-26T21:52:02.865ZINFOnexus (BackgroundTasks): all webhook delivery tasks completed background_task = webhook_deliverator file = nexus/src/app/background/tasks/webhook_deliverator.rs:204 num_receivers = 0
70572025-08-26T21:52:02.865ZDEBGnexus (BackgroundTasks): activation complete background_task = webhook_deliverator elapsed = 4.079341ms iteration = 7 status = {"by_rx":{},"error":null}
70582025-08-26T21:52:02.867ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
70592025-08-26T21:52:02.868ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
70602025-08-26T21:52:02.868ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
70612025-08-26T21:52:02.868ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
70622025-08-26T21:52:02.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
70632025-08-26T21:52:02.868ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
70642025-08-26T21:52:02.868ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
70652025-08-26T21:52:02.868ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
70662025-08-26T21:52:02.868ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
70672025-08-26T21:52:02.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
70682025-08-26T21:52:02.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
70692025-08-26T21:52:02.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
70702025-08-26T21:52:02.869ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
70712025-08-26T21:52:02.869ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
70722025-08-26T21:52:02.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
70732025-08-26T21:52:02.869ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
70742025-08-26T21:52:02.869ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
70752025-08-26T21:52:02.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
70762025-08-26T21:52:02.870ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
70772025-08-26T21:52:02.871ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
70782025-08-26T21:52:02.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
70792025-08-26T21:52:02.872ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
70802025-08-26T21:52:02.872ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
70812025-08-26T21:52:02.872ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
70822025-08-26T21:52:02.872ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
70832025-08-26T21:52:02.872ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 37 reason = Timeout
70842025-08-26T21:52:02.872ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
70852025-08-26T21:52:02.872ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
70862025-08-26T21:52:02.873ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
70872025-08-26T21:52:02.874ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = alert_dispatcher resource = Database result = Ok(())
70882025-08-26T21:52:02.874ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
70892025-08-26T21:52:02.874ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
70902025-08-26T21:52:02.874ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
70912025-08-26T21:52:02.874ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
70922025-08-26T21:52:02.874ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
70932025-08-26T21:52:02.875ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
70942025-08-26T21:52:02.875ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
70952025-08-26T21:52:02.876ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
70962025-08-26T21:52:02.876ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
70972025-08-26T21:52:02.878ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
70982025-08-26T21:52:02.878ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
70992025-08-26T21:52:02.879ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
71002025-08-26T21:52:02.879ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
71012025-08-26T21:52:02.880ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
71022025-08-26T21:52:02.880ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
71032025-08-26T21:52:02.881ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
71042025-08-26T21:52:02.881ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
71052025-08-26T21:52:02.882ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
71062025-08-26T21:52:02.882ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
71072025-08-26T21:52:02.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
71082025-08-26T21:52:02.883ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
71092025-08-26T21:52:02.883ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
71102025-08-26T21:52:02.884ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
71112025-08-26T21:52:02.884ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:550 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
71122025-08-26T21:52:02.884ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 23.301915ms iteration = 13 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
71132025-08-26T21:52:02.885ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
71142025-08-26T21:52:02.886ZDEBGnexus (BackgroundTasks): activation complete background_task = alert_dispatcher elapsed = 25.605477ms iteration = 7 status = {"dispatched":[],"errors":[],"glob_version":"183.0.0","globs_reprocessed":{},"no_receivers":[]}
71152025-08-26T21:52:02.887ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
71162025-08-26T21:52:02.888ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
71172025-08-26T21:52:02.889ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
71182025-08-26T21:52:02.889ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
71192025-08-26T21:52:02.890ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
71202025-08-26T21:52:02.890ZDEBGnexus (BackgroundTasks): activation complete background_task = crdb_node_id_collector elapsed = 29.819902ms iteration = 4 status = {"nsuccess":5}
71212025-08-26T21:52:02.892ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
71222025-08-26T21:52:02.892ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
71232025-08-26T21:52:02.892ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
71242025-08-26T21:52:02.892ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
71252025-08-26T21:52:02.892ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
71262025-08-26T21:52:02.892ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
71272025-08-26T21:52:02.893ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
71282025-08-26T21:52:02.893ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
71292025-08-26T21:52:02.893ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
71302025-08-26T21:52:02.893ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
71312025-08-26T21:52:02.893ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
71322025-08-26T21:52:02.893ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
71332025-08-26T21:52:02.893ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
71342025-08-26T21:52:02.907ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 46.294494ms iteration = 13 status = {"errors":[],"finish_invoked_ok":[]}
71352025-08-26T21:52:02.910ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
71362025-08-26T21:52:02.910ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
71372025-08-26T21:52:02.916ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
71382025-08-26T21:52:02.921ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
71392025-08-26T21:52:02.923ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
71402025-08-26T21:52:02.923ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
71412025-08-26T21:52:02.923ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
71422025-08-26T21:52:02.926ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
71432025-08-26T21:52:02.926ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
71442025-08-26T21:52:02.926ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
71452025-08-26T21:52:02.931ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 70.253294ms iteration = 13 status = {"errors":[],"garbage_collect_requested":[]}
71462025-08-26T21:52:02.932ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
71472025-08-26T21:52:02.932ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:170 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
71482025-08-26T21:52:02.932ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 71.40282ms iteration = 7 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
71492025-08-26T21:52:02.933ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
71502025-08-26T21:52:02.933ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 72.020783ms iteration = 13 status = {"errors":[],"requests_created_ok":[]}
71512025-08-26T21:52:02.933ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
71522025-08-26T21:52:02.933ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
71532025-08-26T21:52:02.934ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
71542025-08-26T21:52:02.934ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
71552025-08-26T21:52:02.935ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
71562025-08-26T21:52:02.936ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
71572025-08-26T21:52:02.936ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
71582025-08-26T21:52:02.938ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
71592025-08-26T21:52:02.938ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
71602025-08-26T21:52:02.938ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
71612025-08-26T21:52:02.938ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
71622025-08-26T21:52:02.938ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
71632025-08-26T21:52:02.938ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
71642025-08-26T21:52:02.938ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
71652025-08-26T21:52:02.939ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
71662025-08-26T21:52:02.939ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
71672025-08-26T21:52:02.939ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
71682025-08-26T21:52:02.940ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1226, tv_nsec: 798863673 } })
71692025-08-26T21:52:02.941ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1205 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
71702025-08-26T21:52:02.944ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 83.408371ms iteration = 13 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
71712025-08-26T21:52:02.947ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
71722025-08-26T21:52:02.947ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 74.744025ms iteration = 37 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
71732025-08-26T21:52:02.951ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
71742025-08-26T21:52:02.951ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
71752025-08-26T21:52:02.951ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
71762025-08-26T21:52:02.951ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
71772025-08-26T21:52:02.951ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
71782025-08-26T21:52:02.952ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
71792025-08-26T21:52:02.952ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
71802025-08-26T21:52:02.953ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 91.961893ms iteration = 14 status = {"physical_disks_added":0}
71812025-08-26T21:52:02.953ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
71822025-08-26T21:52:02.955ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
71832025-08-26T21:52:02.955ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
71842025-08-26T21:52:02.955ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:407
71852025-08-26T21:52:02.955ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 94.684834ms iteration = 13 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
71862025-08-26T21:52:02.956ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
71872025-08-26T21:52:02.958ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
71882025-08-26T21:52:02.958ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
71892025-08-26T21:52:02.959ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
71902025-08-26T21:52:02.961ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
71912025-08-26T21:52:02.962ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: dd0c655a-95c2-48fc-b4c3-9733ed4b0faa, lookup_type: ById(dd0c655a-95c2-48fc-b4c3-9733ed4b0faa) } result = Ok(())
71922025-08-26T21:52:02.962ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
71932025-08-26T21:52:02.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
71942025-08-26T21:52:02.965ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 104.309681ms iteration = 13 status = 0
71952025-08-26T21:52:02.965ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
71962025-08-26T21:52:02.965ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
71972025-08-26T21:52:02.965ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
71982025-08-26T21:52:02.975ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 113.865477ms iteration = 13 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
71992025-08-26T21:52:02.977ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
72002025-08-26T21:52:02.977ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
72012025-08-26T21:52:02.977ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
72022025-08-26T21:52:02.978ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
72032025-08-26T21:52:02.978ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 117.169271ms iteration = 37 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
72042025-08-26T21:52:02.982ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
72052025-08-26T21:52:02.982ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
72062025-08-26T21:52:02.992ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-08-26T21:45:59.866217Z, time_modified: 2025-08-26T21:45:59.866217Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 9396aa55-de6f-4d6f-96db-d28e1c76fa9c, db_silo: Silo { identity: SiloIdentity { id: 9396aa55-de6f-4d6f-96db-d28e1c76fa9c, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-08-26T21:46:05.245521Z, time_modified: 2025-08-26T21:46:05.245521Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "e8529d720e4b0de099753dcdb373abb870af04baacd7c7dd850f5254858081af" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 9396aa55-de6f-4d6f-96db-d28e1c76fa9c, db_silo: Silo { identity: SiloIdentity { id: 9396aa55-de6f-4d6f-96db-d28e1c76fa9c, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-08-26T21:46:05.245521Z, time_modified: 2025-08-26T21:46:05.245521Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "e8529d720e4b0de099753dcdb373abb870af04baacd7c7dd850f5254858081af" }] }) }
72072025-08-26T21:52:02.992ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 158.517684ms iteration = 7 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"9396aa55-de6f-4d6f-96db-d28e1c76fa9c","tls_certs":["e8529d720e4b0de099753dcdb373abb870af04baacd7c7dd850f5254858081af"]}},"default_endpoint":{"silo_id":"9396aa55-de6f-4d6f-96db-d28e1c76fa9c","tls_certs":["e8529d720e4b0de099753dcdb373abb870af04baacd7c7dd850f5254858081af"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
72082025-08-26T21:52:02.994ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
72092025-08-26T21:52:02.994ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:138 found = 0 not_ready_to_be_deleted = 0
72102025-08-26T21:52:02.994ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 133.803957ms iteration = 7 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
72112025-08-26T21:52:03.009ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
72122025-08-26T21:52:03.009ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 8e720f02-dde8-4423-a6e3-91698373805a dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
72132025-08-26T21:52:03.009ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
72142025-08-26T21:52:03.009ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
72152025-08-26T21:52:03.010ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1213 zone_address = fd00:1122:3344:101::2
72162025-08-26T21:52:03.010ZDEBGnexus (MgsClient): client request background_task = nat_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
72172025-08-26T21:52:03.010ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
72182025-08-26T21:52:03.010ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = dfb404ab-acb8-43de-afe8-b429af0c161d dns_zone_name = oxide.test found_so_far = 0 version = 2
72192025-08-26T21:52:03.010ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
72202025-08-26T21:52:03.010ZDEBGnexus (MgsClient): client response background_task = nat_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "7379d580-94b7-4cd7-afd0-7e051e630a39", "content-length": "26", "date": "Tue, 26 Aug 2025 21:52:03 GMT"} })
72212025-08-26T21:52:03.010ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1216 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
72222025-08-26T21:52:03.011ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
72232025-08-26T21:52:03.011ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
72242025-08-26T21:52:03.012ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
72252025-08-26T21:52:03.012ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
72262025-08-26T21:52:03.013ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
72272025-08-26T21:52:03.013ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
72282025-08-26T21:52:03.017ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
72292025-08-26T21:52:03.017ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
72302025-08-26T21:52:03.017ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
72312025-08-26T21:52:03.017ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
72322025-08-26T21:52:03.017ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
72332025-08-26T21:52:03.017ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
72342025-08-26T21:52:03.019ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = dfb404ab-acb8-43de-afe8-b429af0c161d dns_zone_name = oxide.test found_so_far = 4 version = 2
72352025-08-26T21:52:03.019ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
72362025-08-26T21:52:03.019ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2025-08-26 21:46:05.296205 UTC dns_group = external generation = 2
72372025-08-26T21:52:03.019ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 185.392949ms iteration = 7 status = {"generation":2}
72382025-08-26T21:52:03.022ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
72392025-08-26T21:52:03.022ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
72402025-08-26T21:52:03.022ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 161.271496ms iteration = 13 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
72412025-08-26T21:52:03.023ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 8e720f02-dde8-4423-a6e3-91698373805a dns_zone_name = control-plane.oxide.internal found_so_far = 59 version = 1
72422025-08-26T21:52:03.023ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
72432025-08-26T21:52:03.023ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2025-08-26 21:46:04.065217 UTC dns_group = internal generation = 1
72442025-08-26T21:52:03.023ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 189.978842ms iteration = 7 status = {"generation":1}
72452025-08-26T21:52:03.024ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
72462025-08-26T21:52:03.024ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
72472025-08-26T21:52:03.024ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
72482025-08-26T21:52:03.024ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
72492025-08-26T21:52:03.026ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
72502025-08-26T21:52:03.026ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
72512025-08-26T21:52:03.028ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
72522025-08-26T21:52:03.028ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
72532025-08-26T21:52:03.028ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
72542025-08-26T21:52:03.030ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 168.853226ms iteration = 13 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
72552025-08-26T21:52:03.030ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
72562025-08-26T21:52:03.032ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
72572025-08-26T21:52:03.032ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
72582025-08-26T21:52:03.032ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
72592025-08-26T21:52:03.042ZINFOnexus (BackgroundTasks): VPC route manager sled dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
72602025-08-26T21:52:03.042ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
72612025-08-26T21:52:03.042ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "a8bea5ea-830b-488c-82c2-05f1df2786c0", "content-length": "372", "date": "Tue, 26 Aug 2025 21:52:03 GMT"} })
72622025-08-26T21:52:03.042ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
72632025-08-26T21:52:03.043ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
72642025-08-26T21:52:03.091ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
72652025-08-26T21:52:03.091ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
72662025-08-26T21:52:03.092ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "cb3c0d26-87ff-428f-969b-1dd89061a9a4", "date": "Tue, 26 Aug 2025 21:52:03 GMT"} })
72672025-08-26T21:52:03.092ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
72682025-08-26T21:52:03.092ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
72692025-08-26T21:52:03.093ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
72702025-08-26T21:52:03.093ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
72712025-08-26T21:52:03.097ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
72722025-08-26T21:52:03.097ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
72732025-08-26T21:52:03.101ZINFOnexus (BackgroundTasks): VPC route manager sled dd0c655a-95c2-48fc-b4c3-9733ed4b0faa push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
72742025-08-26T21:52:03.101ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
72752025-08-26T21:52:03.101ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "39b8c6ed-da23-40db-98ab-7cbe43903c8e", "date": "Tue, 26 Aug 2025 21:52:03 GMT"} })
72762025-08-26T21:52:03.103ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 242.583718ms iteration = 13 status = {}
72772025-08-26T21:52:03.116ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1213 zone_address = fd00:1122:3344:101::2
72782025-08-26T21:52:03.116ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
72792025-08-26T21:52:03.116ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "1ed4c8f2-07b9-4402-9374-026af6fcab93", "content-length": "26", "date": "Tue, 26 Aug 2025 21:52:03 GMT"} })
72802025-08-26T21:52:03.116ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1216 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
72812025-08-26T21:52:03.117ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
72822025-08-26T21:52:03.121ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
72832025-08-26T21:52:03.135ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1213 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 zone_address = fd00:1122:3344:101::2
72842025-08-26T21:52:03.135ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
72852025-08-26T21:52:03.135ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "cdfbf24b-9e97-42df-b564-c43c48dac51b", "content-length": "26", "date": "Tue, 26 Aug 2025 21:52:03 GMT"} })
72862025-08-26T21:52:03.135ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1216 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
72872025-08-26T21:52:03.137ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
72882025-08-26T21:52:03.138ZINFOnexus (BackgroundTasks): instance reincarnation completed successfully background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:113 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0
72892025-08-26T21:52:03.138ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 277.229179ms iteration = 7 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
72902025-08-26T21:52:03.204ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
72912025-08-26T21:52:03.308ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
72922025-08-26T21:52:03.309ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
72932025-08-26T21:52:03.311ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 477.252398ms iteration = 13 status = null
72942025-08-26T21:52:03.330ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
72952025-08-26T21:52:03.330ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "d4657c0b-a9ef-46a3-8712-dc78750e0ec2", "content-length": "691", "date": "Tue, 26 Aug 2025 21:52:03 GMT"} })
72962025-08-26T21:52:03.330ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
72972025-08-26T21:52:03.330ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
72982025-08-26T21:52:03.330ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
72992025-08-26T21:52:03.332ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 471.639631ms iteration = 13 status = {}
73002025-08-26T21:52:03.349ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
73012025-08-26T21:52:03.350ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "2406fcf9-d182-4063-b6fe-ff7dd7c527db", "date": "Tue, 26 Aug 2025 21:52:03 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
73022025-08-26T21:52:03.350ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
73032025-08-26T21:52:03.352ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "4f31b735-1d98-4e97-8af9-e09c699b360c", "date": "Tue, 26 Aug 2025 21:52:03 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
73042025-08-26T21:52:03.352ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
73052025-08-26T21:52:03.354ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "6c45cf46-2d7a-4e9d-8a2f-b797bb23a6f3", "date": "Tue, 26 Aug 2025 21:52:03 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
73062025-08-26T21:52:03.354ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
73072025-08-26T21:52:03.356ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
73082025-08-26T21:52:03.356ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 522.718654ms iteration = 9 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
73092025-08-26T21:52:03.388ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
73102025-08-26T21:52:03.388ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "7319fd75-a655-4764-a862-e25b18166ecc", "date": "Tue, 26 Aug 2025 21:52:03 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
73112025-08-26T21:52:03.388ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
73122025-08-26T21:52:03.390ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "7a281250-077a-4a2b-bd6e-ca3ae4e57bd9", "date": "Tue, 26 Aug 2025 21:52:03 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
73132025-08-26T21:52:03.390ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
73142025-08-26T21:52:03.390ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 556.459553ms iteration = 13 status = {}
73152025-08-26T21:52:03.391ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
73162025-08-26T21:52:03.391ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 558.06936ms iteration = 9 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
73172025-08-26T21:52:03.420ZDEBGnexus (producer-registration-task): registering as oximeter producer with Nexus registration_count = 2
73182025-08-26T21:52:03.420ZDEBGnexus (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::c]:12221
73192025-08-26T21:52:03.424ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
73202025-08-26T21:52:03.424ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
73212025-08-26T21:52:03.427ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: d24c5c55-3a21-4a91-aeb4-45362e822df6, rack_id: e37a46f2-f789-4ca4-9841-f8c6dd705964, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(88d84f37-a4c8-4abc-af1f-27987b0d39d9) } rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0
73222025-08-26T21:52:03.427ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
73232025-08-26T21:52:03.427ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
73242025-08-26T21:52:03.438ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1832 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
73252025-08-26T21:52:03.438ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1859 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 88d84f37-a4c8-4abc-af1f-27987b0d39d9
73262025-08-26T21:52:03.438ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
73272025-08-26T21:52:03.438ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
73282025-08-26T21:52:03.438ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
73292025-08-26T21:52:03.438ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
73302025-08-26T21:52:03.438ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
73312025-08-26T21:52:03.438ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2006 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
73322025-08-26T21:52:03.438ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:408 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
73332025-08-26T21:52:03.438ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2042 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
73342025-08-26T21:52:03.439ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
73352025-08-26T21:52:03.439ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
73362025-08-26T21:52:03.439ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
73372025-08-26T21:52:03.442ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0
73382025-08-26T21:52:03.442ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
73392025-08-26T21:52:03.452ZDEBGnexus (producer-registration-task): client request body = Some(Body) method = POST uri = http://[fd00:1122:3344:101::c]:12221/metrics/producers
73402025-08-26T21:52:03.452ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:1025 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::c]:50243
73412025-08-26T21:52:03.452ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:50243 req_id = 982e7e84-d592-4442-ba97-010d7333ffdb roles = RoleSet { roles: {} } uri = /metrics/producers
73422025-08-26T21:52:03.453ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:50243 req_id = 982e7e84-d592-4442-ba97-010d7333ffdb resource = Database result = Ok(()) uri = /metrics/producers
73432025-08-26T21:52:03.493ZINFOnexus: assigned collector to new producer collector_id = f843ff0e-eef8-49c4-b428-23e15df1def3 file = nexus/src/app/oximeter.rs:76 producer_id = 5d0959e5-47a6-4bf3-a52b-57950b9d900d
73442025-08-26T21:52:03.493ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 40784 local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:50243 req_id = 982e7e84-d592-4442-ba97-010d7333ffdb response_code = 201 uri = /metrics/producers
73452025-08-26T21:52:03.493ZDEBGnexus (producer-registration-task): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::c]:12221/metrics/producers", status: 201, headers: {"content-type": "application/json", "x-request-id": "982e7e84-d592-4442-ba97-010d7333ffdb", "content-length": "41", "date": "Tue, 26 Aug 2025 21:52:03 GMT"} })
73462025-08-26T21:52:03.495ZDEBGnexus (producer-registration-task): successfully registered with Nexus, pausing until time to renew lease lease_duration = 600s wait_period = 150s
73472025-08-26T21:52:03.544ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "2f86c774-6a6d-4a46-9959-d3b9b22ca095", "date": "Tue, 26 Aug 2025 21:52:03 GMT"} })
73482025-08-26T21:52:03.544ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch = Switch0
73492025-08-26T21:52:03.544ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
73502025-08-26T21:52:03.545ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "d4b36940-d523-45ae-810a-fc1602195463", "content-length": "682", "date": "Tue, 26 Aug 2025 21:52:03 GMT"} })
73512025-08-26T21:52:03.545ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
73522025-08-26T21:52:03.545ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
73532025-08-26T21:52:03.549ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
73542025-08-26T21:52:03.549ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
73552025-08-26T21:52:03.552ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
73562025-08-26T21:52:03.552ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
73572025-08-26T21:52:03.554ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
73582025-08-26T21:52:03.554ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
73592025-08-26T21:52:03.562ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 701.120904ms iteration = 13 status = {}
73602025-08-26T21:52:05.353ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 325 local_addr = [fd00:1122:3344:101::c]:42016 method = GET remote_addr = [fd00:1122:3344:101::d]:37550 req_id = 8a3ab5ed-eb25-4e46-a8b8-721d0c0a9f2e response_code = 200 uri = /5d0959e5-47a6-4bf3-a52b-57950b9d900d
73612025-08-26T21:52:07.848ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 74 reason = Timeout
73622025-08-26T21:52:07.848ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
73632025-08-26T21:52:07.848ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
73642025-08-26T21:52:07.850ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
73652025-08-26T21:52:07.853ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
73662025-08-26T21:52:07.853ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
73672025-08-26T21:52:07.853ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
73682025-08-26T21:52:07.856ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
73692025-08-26T21:52:07.856ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 8.087863ms iteration = 74 status = {"chicken_switches_updated":false}
73702025-08-26T21:52:12.847ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 75 reason = Timeout
73712025-08-26T21:52:12.847ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
73722025-08-26T21:52:12.847ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
73732025-08-26T21:52:12.849ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
73742025-08-26T21:52:12.852ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
73752025-08-26T21:52:12.852ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
73762025-08-26T21:52:12.852ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
73772025-08-26T21:52:12.854ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
73782025-08-26T21:52:12.854ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.09249ms iteration = 75 status = {"chicken_switches_updated":false}
73792025-08-26T21:52:12.860ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 38 reason = Timeout
73802025-08-26T21:52:12.860ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
73812025-08-26T21:52:12.860ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
73822025-08-26T21:52:12.872ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 38 reason = Timeout
73832025-08-26T21:52:12.873ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
73842025-08-26T21:52:12.873ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
73852025-08-26T21:52:12.874ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
73862025-08-26T21:52:12.877ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
73872025-08-26T21:52:12.877ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
73882025-08-26T21:52:12.877ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
73892025-08-26T21:52:12.879ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
73902025-08-26T21:52:12.880ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
73912025-08-26T21:52:12.881ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
73922025-08-26T21:52:12.883ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
73932025-08-26T21:52:12.883ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
73942025-08-26T21:52:12.884ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
73952025-08-26T21:52:12.896ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
73962025-08-26T21:52:12.896ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 23.771236ms iteration = 38 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
73972025-08-26T21:52:12.896ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
73982025-08-26T21:52:12.897ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
73992025-08-26T21:52:12.898ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 37.813083ms iteration = 38 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
74002025-08-26T21:52:15.353ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 318 local_addr = [fd00:1122:3344:101::c]:42016 method = GET remote_addr = [fd00:1122:3344:101::d]:37550 req_id = 60061bb0-c28e-436f-88cd-c648bb0f7a3e response_code = 200 uri = /5d0959e5-47a6-4bf3-a52b-57950b9d900d
74012025-08-26T21:52:17.848ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 76 reason = Timeout
74022025-08-26T21:52:17.848ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
74032025-08-26T21:52:17.848ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
74042025-08-26T21:52:17.850ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
74052025-08-26T21:52:17.853ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
74062025-08-26T21:52:17.853ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
74072025-08-26T21:52:17.853ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
74082025-08-26T21:52:17.855ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
74092025-08-26T21:52:17.855ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.394927ms iteration = 76 status = {"chicken_switches_updated":false}
74102025-08-26T21:52:22.847ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 77 reason = Timeout
74112025-08-26T21:52:22.847ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
74122025-08-26T21:52:22.848ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
74132025-08-26T21:52:22.850ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
74142025-08-26T21:52:22.852ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
74152025-08-26T21:52:22.852ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
74162025-08-26T21:52:22.853ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
74172025-08-26T21:52:22.854ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
74182025-08-26T21:52:22.854ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.236533ms iteration = 77 status = {"chicken_switches_updated":false}
74192025-08-26T21:52:22.860ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 39 reason = Timeout
74202025-08-26T21:52:22.860ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
74212025-08-26T21:52:22.860ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
74222025-08-26T21:52:22.862ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
74232025-08-26T21:52:22.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
74242025-08-26T21:52:22.864ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.808816ms iteration = 39 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
74252025-08-26T21:52:22.873ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 39 reason = Timeout
74262025-08-26T21:52:22.873ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
74272025-08-26T21:52:22.873ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
74282025-08-26T21:52:22.874ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
74292025-08-26T21:52:22.877ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
74302025-08-26T21:52:22.877ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
74312025-08-26T21:52:22.877ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
74322025-08-26T21:52:22.879ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
74332025-08-26T21:52:22.879ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
74342025-08-26T21:52:22.881ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
74352025-08-26T21:52:22.884ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
74362025-08-26T21:52:22.884ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
74372025-08-26T21:52:22.884ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
74382025-08-26T21:52:22.896ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
74392025-08-26T21:52:22.896ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 23.177612ms iteration = 39 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
74402025-08-26T21:52:25.353ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 449 local_addr = [fd00:1122:3344:101::c]:42016 method = GET remote_addr = [fd00:1122:3344:101::d]:37550 req_id = 59ffb990-c89d-4ab8-b326-b8507e046ba5 response_code = 200 uri = /5d0959e5-47a6-4bf3-a52b-57950b9d900d
74412025-08-26T21:52:27.848ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 78 reason = Timeout
74422025-08-26T21:52:27.848ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
74432025-08-26T21:52:27.848ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
74442025-08-26T21:52:27.850ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
74452025-08-26T21:52:27.852ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
74462025-08-26T21:52:27.853ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
74472025-08-26T21:52:27.853ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
74482025-08-26T21:52:27.855ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
74492025-08-26T21:52:27.855ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.020388ms iteration = 78 status = {"chicken_switches_updated":false}
74502025-08-26T21:52:30.345ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:1025 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::d]:64173
74512025-08-26T21:52:30.345ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:64173 req_id = 0b789064-5847-4c7a-97ef-b48d87e577fd roles = RoleSet { roles: {} } uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?sort_by=id_ascending
74522025-08-26T21:52:30.346ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:64173 req_id = 0b789064-5847-4c7a-97ef-b48d87e577fd resource = Database result = Ok(()) uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?sort_by=id_ascending
74532025-08-26T21:52:30.348ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 3180 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:64173 req_id = 0b789064-5847-4c7a-97ef-b48d87e577fd response_code = 200 uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?sort_by=id_ascending
74542025-08-26T21:52:30.349ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:64173 req_id = 94e9e636-e66b-44d9-9a6d-ee4a93899885 roles = RoleSet { roles: {} } uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMzhjYTg4YS1lZGE1LTQwZGEtOGE4MS1hN2UzMzdhMmM4ZmIifX0%3D
74552025-08-26T21:52:30.349ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:64173 req_id = 94e9e636-e66b-44d9-9a6d-ee4a93899885 resource = Database result = Ok(()) uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMzhjYTg4YS1lZGE1LTQwZGEtOGE4MS1hN2UzMzdhMmM4ZmIifX0%3D
74562025-08-26T21:52:30.352ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 3110 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:64173 req_id = 94e9e636-e66b-44d9-9a6d-ee4a93899885 response_code = 200 uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMzhjYTg4YS1lZGE1LTQwZGEtOGE4MS1hN2UzMzdhMmM4ZmIifX0%3D
74572025-08-26T21:52:32.833ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 14 reason = Timeout
74582025-08-26T21:52:32.833ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 14 reason = Timeout
74592025-08-26T21:52:32.833ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
74602025-08-26T21:52:32.833ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 14 reason = Timeout
74612025-08-26T21:52:32.833ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
74622025-08-26T21:52:32.833ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
74632025-08-26T21:52:32.833ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
74642025-08-26T21:52:32.836ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 2.701421ms iteration = 14 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
74652025-08-26T21:52:32.836ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1256, tv_nsec: 694529101 } })
74662025-08-26T21:52:32.836ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1256, tv_nsec: 694583542 } })
74672025-08-26T21:52:32.836ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1205
74682025-08-26T21:52:32.868ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1213 zone_address = fd00:1122:3344:101::2
74692025-08-26T21:52:32.868ZDEBGnexus (MgsClient): client request background_task = nat_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
74702025-08-26T21:52:32.868ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 14 reason = Timeout
74712025-08-26T21:52:32.868ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
74722025-08-26T21:52:32.868ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 14 reason = Timeout
74732025-08-26T21:52:32.868ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
74742025-08-26T21:52:32.868ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 14 reason = Timeout
74752025-08-26T21:52:32.868ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
74762025-08-26T21:52:32.868ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 14 reason = Timeout
74772025-08-26T21:52:32.868ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1205
74782025-08-26T21:52:32.868ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 14 reason = Timeout
74792025-08-26T21:52:32.868ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
74802025-08-26T21:52:32.868ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
74812025-08-26T21:52:32.868ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 15 reason = Timeout
74822025-08-26T21:52:32.868ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
74832025-08-26T21:52:32.868ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 14 reason = Timeout
74842025-08-26T21:52:32.868ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 14 reason = Timeout
74852025-08-26T21:52:32.868ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
74862025-08-26T21:52:32.868ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
74872025-08-26T21:52:32.868ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 40 reason = Timeout
74882025-08-26T21:52:32.868ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
74892025-08-26T21:52:32.868ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 14 reason = Timeout
74902025-08-26T21:52:32.868ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 14 reason = Timeout
74912025-08-26T21:52:32.868ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
74922025-08-26T21:52:32.868ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
74932025-08-26T21:52:32.868ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 14 reason = Timeout
74942025-08-26T21:52:32.868ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
74952025-08-26T21:52:32.868ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
74962025-08-26T21:52:32.868ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 14 reason = Timeout
74972025-08-26T21:52:32.868ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
74982025-08-26T21:52:32.868ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 14 reason = Timeout
74992025-08-26T21:52:32.868ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
75002025-08-26T21:52:32.868ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 14 reason = Timeout
75012025-08-26T21:52:32.868ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
75022025-08-26T21:52:32.868ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
75032025-08-26T21:52:32.868ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 14 reason = Timeout
75042025-08-26T21:52:32.868ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 9.98\u{b5}s iteration = 14 status = {"disabled":true,"errors":[],"sps":[]}
75052025-08-26T21:52:32.868ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 79 reason = Timeout
75062025-08-26T21:52:32.868ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
75072025-08-26T21:52:32.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
75082025-08-26T21:52:32.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
75092025-08-26T21:52:32.868ZDEBGnexus (MgsClient): client response background_task = nat_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "8ff6036e-3adf-4831-a093-7111c8217601", "content-length": "26", "date": "Tue, 26 Aug 2025 21:52:32 GMT"} })
75102025-08-26T21:52:32.868ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1216 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
75112025-08-26T21:52:32.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
75122025-08-26T21:52:32.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
75132025-08-26T21:52:32.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
75142025-08-26T21:52:32.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
75152025-08-26T21:52:32.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
75162025-08-26T21:52:32.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
75172025-08-26T21:52:32.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
75182025-08-26T21:52:32.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
75192025-08-26T21:52:32.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
75202025-08-26T21:52:32.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
75212025-08-26T21:52:32.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
75222025-08-26T21:52:32.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
75232025-08-26T21:52:32.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
75242025-08-26T21:52:32.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
75252025-08-26T21:52:32.870ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
75262025-08-26T21:52:32.870ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
75272025-08-26T21:52:32.870ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
75282025-08-26T21:52:32.870ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 2.584428ms iteration = 14 status = {"errors":[],"finish_invoked_ok":[]}
75292025-08-26T21:52:32.871ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 2.540477ms iteration = 14 status = {"errors":[],"garbage_collect_requested":[]}
75302025-08-26T21:52:32.871ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
75312025-08-26T21:52:32.871ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
75322025-08-26T21:52:32.871ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
75332025-08-26T21:52:32.871ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
75342025-08-26T21:52:32.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
75352025-08-26T21:52:32.871ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
75362025-08-26T21:52:32.872ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
75372025-08-26T21:52:32.872ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
75382025-08-26T21:52:32.872ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
75392025-08-26T21:52:32.872ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
75402025-08-26T21:52:32.872ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 40 reason = Timeout
75412025-08-26T21:52:32.872ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
75422025-08-26T21:52:32.873ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
75432025-08-26T21:52:32.874ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 5.76461ms iteration = 14 status = {"errors":[],"requests_created_ok":[]}
75442025-08-26T21:52:32.874ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
75452025-08-26T21:52:32.875ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
75462025-08-26T21:52:32.875ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
75472025-08-26T21:52:32.875ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
75482025-08-26T21:52:32.875ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
75492025-08-26T21:52:32.875ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
75502025-08-26T21:52:32.875ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
75512025-08-26T21:52:32.876ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
75522025-08-26T21:52:32.876ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
75532025-08-26T21:52:32.876ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
75542025-08-26T21:52:32.876ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
75552025-08-26T21:52:32.876ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
75562025-08-26T21:52:32.876ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
75572025-08-26T21:52:32.876ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
75582025-08-26T21:52:32.876ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
75592025-08-26T21:52:32.876ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
75602025-08-26T21:52:32.876ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
75612025-08-26T21:52:32.877ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
75622025-08-26T21:52:32.877ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
75632025-08-26T21:52:32.877ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
75642025-08-26T21:52:32.877ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
75652025-08-26T21:52:32.877ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
75662025-08-26T21:52:32.878ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
75672025-08-26T21:52:32.878ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
75682025-08-26T21:52:32.878ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 10.255081ms iteration = 79 status = {"chicken_switches_updated":false}
75692025-08-26T21:52:32.879ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
75702025-08-26T21:52:32.879ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
75712025-08-26T21:52:32.879ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
75722025-08-26T21:52:32.879ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
75732025-08-26T21:52:32.879ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
75742025-08-26T21:52:32.879ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
75752025-08-26T21:52:32.879ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
75762025-08-26T21:52:32.879ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
75772025-08-26T21:52:32.880ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
75782025-08-26T21:52:32.881ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
75792025-08-26T21:52:32.881ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
75802025-08-26T21:52:32.881ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
75812025-08-26T21:52:32.881ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
75822025-08-26T21:52:32.881ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
75832025-08-26T21:52:32.882ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
75842025-08-26T21:52:32.883ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
75852025-08-26T21:52:32.883ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
75862025-08-26T21:52:32.884ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
75872025-08-26T21:52:32.885ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
75882025-08-26T21:52:32.885ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
75892025-08-26T21:52:32.885ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
75902025-08-26T21:52:32.889ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
75912025-08-26T21:52:32.890ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
75922025-08-26T21:52:32.892ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
75932025-08-26T21:52:32.895ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
75942025-08-26T21:52:32.895ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
75952025-08-26T21:52:32.895ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
75962025-08-26T21:52:32.897ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
75972025-08-26T21:52:32.899ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
75982025-08-26T21:52:32.899ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
75992025-08-26T21:52:32.899ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
76002025-08-26T21:52:32.905ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
76012025-08-26T21:52:32.905ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 32.672357ms iteration = 40 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
76022025-08-26T21:52:32.917ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
76032025-08-26T21:52:32.919ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
76042025-08-26T21:52:32.919ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
76052025-08-26T21:52:32.920ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
76062025-08-26T21:52:32.922ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1256, tv_nsec: 780963770 } })
76072025-08-26T21:52:32.923ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1205 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
76082025-08-26T21:52:32.927ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 59.234915ms iteration = 40 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
76092025-08-26T21:52:32.928ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
76102025-08-26T21:52:32.928ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 60.211587ms iteration = 14 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
76112025-08-26T21:52:32.928ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
76122025-08-26T21:52:32.929ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
76132025-08-26T21:52:32.929ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
76142025-08-26T21:52:32.929ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
76152025-08-26T21:52:32.930ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
76162025-08-26T21:52:32.930ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
76172025-08-26T21:52:32.930ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
76182025-08-26T21:52:32.931ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
76192025-08-26T21:52:32.931ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:550 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
76202025-08-26T21:52:32.931ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 63.356078ms iteration = 14 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
76212025-08-26T21:52:32.932ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 63.688396ms iteration = 14 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
76222025-08-26T21:52:32.932ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
76232025-08-26T21:52:32.932ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
76242025-08-26T21:52:32.933ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
76252025-08-26T21:52:32.934ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
76262025-08-26T21:52:32.937ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: dd0c655a-95c2-48fc-b4c3-9733ed4b0faa, lookup_type: ById(dd0c655a-95c2-48fc-b4c3-9733ed4b0faa) } result = Ok(())
76272025-08-26T21:52:32.937ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
76282025-08-26T21:52:32.937ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
76292025-08-26T21:52:32.939ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
76302025-08-26T21:52:32.939ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
76312025-08-26T21:52:32.939ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
76322025-08-26T21:52:32.939ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
76332025-08-26T21:52:32.941ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 73.483566ms iteration = 14 status = 0
76342025-08-26T21:52:32.955ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
76352025-08-26T21:52:32.955ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
76362025-08-26T21:52:32.957ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 124.467886ms iteration = 14 status = null
76372025-08-26T21:52:32.970ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1213 zone_address = fd00:1122:3344:101::2
76382025-08-26T21:52:32.970ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
76392025-08-26T21:52:32.970ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "7d667091-4618-41a1-9eec-fc3dc80c78e9", "content-length": "26", "date": "Tue, 26 Aug 2025 21:52:32 GMT"} })
76402025-08-26T21:52:32.970ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1216 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
76412025-08-26T21:52:32.972ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
76422025-08-26T21:52:32.974ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 105.908448ms iteration = 14 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
76432025-08-26T21:52:32.977ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
76442025-08-26T21:52:32.978ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
76452025-08-26T21:52:32.978ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
76462025-08-26T21:52:32.978ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 109.664892ms iteration = 14 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
76472025-08-26T21:52:32.979ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
76482025-08-26T21:52:32.979ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
76492025-08-26T21:52:32.980ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
76502025-08-26T21:52:33.008ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
76512025-08-26T21:52:33.008ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "cc0c91e8-2f8d-47fa-83fd-ab56a378ad08", "content-length": "691", "date": "Tue, 26 Aug 2025 21:52:33 GMT"} })
76522025-08-26T21:52:33.008ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
76532025-08-26T21:52:33.008ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
76542025-08-26T21:52:33.008ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
76552025-08-26T21:52:33.010ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 141.837028ms iteration = 14 status = {}
76562025-08-26T21:52:33.016ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 148.090798ms iteration = 15 status = {"physical_disks_added":0}
76572025-08-26T21:52:33.017ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:407
76582025-08-26T21:52:33.017ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 148.985518ms iteration = 14 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
76592025-08-26T21:52:33.033ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1213 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 zone_address = fd00:1122:3344:101::2
76602025-08-26T21:52:33.033ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
76612025-08-26T21:52:33.033ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "eb138754-e9cf-4a9a-aa12-72ecbf9c6b36", "content-length": "26", "date": "Tue, 26 Aug 2025 21:52:32 GMT"} })
76622025-08-26T21:52:33.033ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1216 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
76632025-08-26T21:52:33.035ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
76642025-08-26T21:52:33.061ZINFOnexus (BackgroundTasks): VPC route manager sled dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
76652025-08-26T21:52:33.061ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
76662025-08-26T21:52:33.061ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "73320c45-4dea-4349-bbff-a279123b5f95", "content-length": "372", "date": "Tue, 26 Aug 2025 21:52:33 GMT"} })
76672025-08-26T21:52:33.061ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
76682025-08-26T21:52:33.061ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
76692025-08-26T21:52:33.065ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 232.582003ms iteration = 14 status = {}
76702025-08-26T21:52:33.111ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
76712025-08-26T21:52:33.111ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
76722025-08-26T21:52:33.111ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "6002a84e-2d55-45bf-8cb1-2e224090c841", "date": "Tue, 26 Aug 2025 21:52:33 GMT"} })
76732025-08-26T21:52:33.111ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
76742025-08-26T21:52:33.112ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
76752025-08-26T21:52:33.114ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
76762025-08-26T21:52:33.114ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
76772025-08-26T21:52:33.117ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
76782025-08-26T21:52:33.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
76792025-08-26T21:52:33.120ZINFOnexus (BackgroundTasks): VPC route manager sled dd0c655a-95c2-48fc-b4c3-9733ed4b0faa push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
76802025-08-26T21:52:33.120ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
76812025-08-26T21:52:33.121ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "06eb3714-fa62-4b47-b26c-61a7dc3c9f92", "date": "Tue, 26 Aug 2025 21:52:33 GMT"} })
76822025-08-26T21:52:33.122ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 254.238041ms iteration = 14 status = {}
76832025-08-26T21:52:33.143ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
76842025-08-26T21:52:33.144ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
76852025-08-26T21:52:33.146ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: d24c5c55-3a21-4a91-aeb4-45362e822df6, rack_id: e37a46f2-f789-4ca4-9841-f8c6dd705964, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(88d84f37-a4c8-4abc-af1f-27987b0d39d9) } rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0
76862025-08-26T21:52:33.146ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
76872025-08-26T21:52:33.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
76882025-08-26T21:52:33.158ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1832 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
76892025-08-26T21:52:33.158ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1859 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 88d84f37-a4c8-4abc-af1f-27987b0d39d9
76902025-08-26T21:52:33.158ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
76912025-08-26T21:52:33.158ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
76922025-08-26T21:52:33.158ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
76932025-08-26T21:52:33.158ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
76942025-08-26T21:52:33.158ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
76952025-08-26T21:52:33.158ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2006 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
76962025-08-26T21:52:33.158ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:408 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
76972025-08-26T21:52:33.158ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2042 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
76982025-08-26T21:52:33.159ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
76992025-08-26T21:52:33.159ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
77002025-08-26T21:52:33.159ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
77012025-08-26T21:52:33.160ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0
77022025-08-26T21:52:33.161ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
77032025-08-26T21:52:33.271ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "2d0382fc-481b-4a3b-ad12-131fb4bbc6fc", "date": "Tue, 26 Aug 2025 21:52:33 GMT"} })
77042025-08-26T21:52:33.271ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch = Switch0
77052025-08-26T21:52:33.271ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
77062025-08-26T21:52:33.271ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "c344d9df-7208-40f1-9a7d-fb472e7d54f3", "content-length": "682", "date": "Tue, 26 Aug 2025 21:52:33 GMT"} })
77072025-08-26T21:52:33.271ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
77082025-08-26T21:52:33.272ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
77092025-08-26T21:52:33.275ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
77102025-08-26T21:52:33.275ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
77112025-08-26T21:52:33.279ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
77122025-08-26T21:52:33.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
77132025-08-26T21:52:33.281ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
77142025-08-26T21:52:33.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
77152025-08-26T21:52:33.288ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 420.357206ms iteration = 14 status = {}
77162025-08-26T21:52:35.352ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 338 local_addr = [fd00:1122:3344:101::c]:42016 method = GET remote_addr = [fd00:1122:3344:101::d]:37550 req_id = be5e9184-eddf-45c9-a724-6015c43f9146 response_code = 200 uri = /5d0959e5-47a6-4bf3-a52b-57950b9d900d
77172025-08-26T21:52:37.868ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 80 reason = Timeout
77182025-08-26T21:52:37.868ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
77192025-08-26T21:52:37.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
77202025-08-26T21:52:37.870ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
77212025-08-26T21:52:37.873ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
77222025-08-26T21:52:37.873ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
77232025-08-26T21:52:37.874ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
77242025-08-26T21:52:37.876ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
77252025-08-26T21:52:37.876ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.176992ms iteration = 80 status = {"chicken_switches_updated":false}
77262025-08-26T21:52:42.868ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 41 reason = Timeout
77272025-08-26T21:52:42.868ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 81 reason = Timeout
77282025-08-26T21:52:42.869ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
77292025-08-26T21:52:42.869ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
77302025-08-26T21:52:42.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
77312025-08-26T21:52:42.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
77322025-08-26T21:52:42.871ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
77332025-08-26T21:52:42.871ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
77342025-08-26T21:52:42.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
77352025-08-26T21:52:42.871ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 41 reason = Timeout
77362025-08-26T21:52:42.871ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
77372025-08-26T21:52:42.872ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
77382025-08-26T21:52:42.873ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
77392025-08-26T21:52:42.874ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
77402025-08-26T21:52:42.874ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
77412025-08-26T21:52:42.874ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
77422025-08-26T21:52:42.876ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
77432025-08-26T21:52:42.876ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.541539ms iteration = 81 status = {"chicken_switches_updated":false}
77442025-08-26T21:52:42.876ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
77452025-08-26T21:52:42.876ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
77462025-08-26T21:52:42.877ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
77472025-08-26T21:52:42.878ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
77482025-08-26T21:52:42.878ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
77492025-08-26T21:52:42.879ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
77502025-08-26T21:52:42.882ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
77512025-08-26T21:52:42.882ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
77522025-08-26T21:52:42.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
77532025-08-26T21:52:42.898ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
77542025-08-26T21:52:42.898ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 26.493208ms iteration = 41 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
77552025-08-26T21:52:42.907ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 38.1195ms iteration = 41 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
77562025-08-26T21:52:45.352ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 312 local_addr = [fd00:1122:3344:101::c]:42016 method = GET remote_addr = [fd00:1122:3344:101::d]:37550 req_id = 59cc3656-4e2b-422a-a71f-3ad1bbaae657 response_code = 200 uri = /5d0959e5-47a6-4bf3-a52b-57950b9d900d
77572025-08-26T21:52:47.868ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 82 reason = Timeout
77582025-08-26T21:52:47.868ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
77592025-08-26T21:52:47.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
77602025-08-26T21:52:47.870ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
77612025-08-26T21:52:47.873ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
77622025-08-26T21:52:47.873ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
77632025-08-26T21:52:47.873ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
77642025-08-26T21:52:47.875ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
77652025-08-26T21:52:47.875ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 6.747012ms iteration = 82 status = {"chicken_switches_updated":false}
77662025-08-26T21:52:52.868ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 42 reason = Timeout
77672025-08-26T21:52:52.869ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
77682025-08-26T21:52:52.869ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 83 reason = Timeout
77692025-08-26T21:52:52.869ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
77702025-08-26T21:52:52.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
77712025-08-26T21:52:52.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
77722025-08-26T21:52:52.871ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
77732025-08-26T21:52:52.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
77742025-08-26T21:52:52.871ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 42 reason = Timeout
77752025-08-26T21:52:52.871ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
77762025-08-26T21:52:52.871ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
77772025-08-26T21:52:52.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
77782025-08-26T21:52:52.872ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.820216ms iteration = 42 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
77792025-08-26T21:52:52.873ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
77802025-08-26T21:52:52.874ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
77812025-08-26T21:52:52.874ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
77822025-08-26T21:52:52.874ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
77832025-08-26T21:52:52.876ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
77842025-08-26T21:52:52.876ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
77852025-08-26T21:52:52.876ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
77862025-08-26T21:52:52.876ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
77872025-08-26T21:52:52.876ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.593051ms iteration = 83 status = {"chicken_switches_updated":false}
77882025-08-26T21:52:52.877ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
77892025-08-26T21:52:52.877ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
77902025-08-26T21:52:52.878ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
77912025-08-26T21:52:52.881ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
77922025-08-26T21:52:52.881ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
77932025-08-26T21:52:52.881ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
77942025-08-26T21:52:52.896ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
77952025-08-26T21:52:52.896ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 24.723407ms iteration = 42 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
77962025-08-26T21:52:55.352ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 312 local_addr = [fd00:1122:3344:101::c]:42016 method = GET remote_addr = [fd00:1122:3344:101::d]:37550 req_id = 11b66f74-3a64-42d4-8490-d9eabf89f85e response_code = 200 uri = /5d0959e5-47a6-4bf3-a52b-57950b9d900d
77972025-08-26T21:52:57.869ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 84 reason = Timeout
77982025-08-26T21:52:57.869ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
77992025-08-26T21:52:57.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
78002025-08-26T21:52:57.870ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
78012025-08-26T21:52:57.873ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
78022025-08-26T21:52:57.873ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
78032025-08-26T21:52:57.873ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
78042025-08-26T21:52:57.875ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
78052025-08-26T21:52:57.875ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 6.506037ms iteration = 84 status = {"chicken_switches_updated":false}
78062025-08-26T21:53:02.832ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 8 reason = Timeout
78072025-08-26T21:53:02.832ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 8 reason = Timeout
78082025-08-26T21:53:02.832ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
78092025-08-26T21:53:02.832ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
78102025-08-26T21:53:02.832ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 15 reason = Timeout
78112025-08-26T21:53:02.832ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 15 reason = Timeout
78122025-08-26T21:53:02.832ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 10 reason = Timeout
78132025-08-26T21:53:02.832ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
78142025-08-26T21:53:02.832ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 10 reason = Timeout
78152025-08-26T21:53:02.832ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 15 reason = Timeout
78162025-08-26T21:53:02.832ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
78172025-08-26T21:53:02.832ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 8 reason = Timeout
78182025-08-26T21:53:02.833ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-08-26 21:43:02.832955125 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
78192025-08-26T21:53:02.833ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 8 reason = Timeout
78202025-08-26T21:53:02.833ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
78212025-08-26T21:53:02.833ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 8 reason = Timeout
78222025-08-26T21:53:02.833ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
78232025-08-26T21:53:02.833ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 8 reason = Timeout
78242025-08-26T21:53:02.833ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
78252025-08-26T21:53:02.833ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
78262025-08-26T21:53:02.833ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
78272025-08-26T21:53:02.833ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
78282025-08-26T21:53:02.833ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
78292025-08-26T21:53:02.833ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
78302025-08-26T21:53:02.833ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("852fd1a3-de9a-47e8-a667-5e8dab880ad3.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("c68f1501-294c-4874-92ca-8fc0546a5870.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("d741a72f-8745-4723-9e80-e1e72987266b.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("d741a72f-8745-4723-9e80-e1e72987266b.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:3::1)) }], valid_until: Instant { tv_sec: 1286, tv_nsec: 692435534 } })
78312025-08-26T21:53:02.833ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
78322025-08-26T21:53:02.833ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("60abf731-31e1-4111-a3e2-65da89eaed87.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("84c849eb-0949-41d9-ad02-9a5509144564.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("84c849eb-0949-41d9-ad02-9a5509144564.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::8)) }], valid_until: Instant { tv_sec: 1286, tv_nsec: 692578008 } })
78332025-08-26T21:53:02.833ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
78342025-08-26T21:53:02.833ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
78352025-08-26T21:53:02.834ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
78362025-08-26T21:53:02.834ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } dns_group = internal
78372025-08-26T21:53:02.834ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.112315ms iteration = 8 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
78382025-08-26T21:53:02.834ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.33434ms iteration = 8 status = {"addresses":["[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:3::1]:5353"]}
78392025-08-26T21:53:02.834ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
78402025-08-26T21:53:02.834ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
78412025-08-26T21:53:02.834ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
78422025-08-26T21:53:02.835ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
78432025-08-26T21:53:02.835ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 9 reason = Timeout
78442025-08-26T21:53:02.835ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:91 reason = blueprint disabled target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8
78452025-08-26T21:53:02.835ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 14.29\u{b5}s iteration = 9 status = {"enabled":false,"target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8"}
78462025-08-26T21:53:02.835ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1286, tv_nsec: 694713156 } })
78472025-08-26T21:53:02.836ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1205
78482025-08-26T21:53:02.836ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
78492025-08-26T21:53:02.836ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-08-26 21:43:02.832955125 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
78502025-08-26T21:53:02.836ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 3.196492ms iteration = 8 status = {"expiration":"2025-08-26T21:43:02.832955125Z","pruned":[]}
78512025-08-26T21:53:02.836ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
78522025-08-26T21:53:02.836ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1286, tv_nsec: 695336070 } })
78532025-08-26T21:53:02.836ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1205
78542025-08-26T21:53:02.837ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 4.503972ms iteration = 15 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
78552025-08-26T21:53:02.838ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
78562025-08-26T21:53:02.838ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
78572025-08-26T21:53:02.838ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
78582025-08-26T21:53:02.840ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
78592025-08-26T21:53:02.840ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
78602025-08-26T21:53:02.840ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
78612025-08-26T21:53:02.840ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
78622025-08-26T21:53:02.841ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
78632025-08-26T21:53:02.844ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
78642025-08-26T21:53:02.844ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
78652025-08-26T21:53:02.844ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
78662025-08-26T21:53:02.860ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 10 reason = Timeout
78672025-08-26T21:53:02.860ZDEBGnexus (BackgroundTasks): blueprint planning disabled, doing nothing background_task = blueprint_planner
78682025-08-26T21:53:02.860ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 8 reason = Timeout
78692025-08-26T21:53:02.860ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 11.74\u{b5}s iteration = 10 status = "Disabled"
78702025-08-26T21:53:02.860ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:53
78712025-08-26T21:53:02.860ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 8 reason = Timeout
78722025-08-26T21:53:02.860ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
78732025-08-26T21:53:02.860ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
78742025-08-26T21:53:02.860ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 8 reason = Timeout
78752025-08-26T21:53:02.860ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
78762025-08-26T21:53:02.860ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
78772025-08-26T21:53:02.860ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
78782025-08-26T21:53:02.860ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 8 reason = Timeout
78792025-08-26T21:53:02.860ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
78802025-08-26T21:53:02.860ZDEBGnexus (BackgroundTasks): activating background_task = webhook_deliverator iteration = 8 reason = Timeout
78812025-08-26T21:53:02.860ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
78822025-08-26T21:53:02.860ZDEBGnexus (BackgroundTasks): activating background_task = alert_dispatcher iteration = 8 reason = Timeout
78832025-08-26T21:53:02.860ZDEBGnexus (BackgroundTasks): roles background_task = webhook_deliverator roles = RoleSet { roles: {} }
78842025-08-26T21:53:02.860ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
78852025-08-26T21:53:02.860ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = webhook_deliverator resource = Database result = Ok(())
78862025-08-26T21:53:02.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
78872025-08-26T21:53:02.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = alert_dispatcher resource = Database result = Ok(())
78882025-08-26T21:53:02.862ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
78892025-08-26T21:53:02.863ZINFOnexus (BackgroundTasks): all webhook delivery tasks completed background_task = webhook_deliverator file = nexus/src/app/background/tasks/webhook_deliverator.rs:204 num_receivers = 0
78902025-08-26T21:53:02.863ZDEBGnexus (BackgroundTasks): activation complete background_task = webhook_deliverator elapsed = 2.265811ms iteration = 8 status = {"by_rx":{},"error":null}
78912025-08-26T21:53:02.863ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 2.320063ms iteration = 8 status = {"errors":[],"found_port_ok":[]}
78922025-08-26T21:53:02.864ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
78932025-08-26T21:53:02.864ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:170 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
78942025-08-26T21:53:02.864ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 4.291066ms iteration = 8 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
78952025-08-26T21:53:02.864ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
78962025-08-26T21:53:02.864ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
78972025-08-26T21:53:02.864ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
78982025-08-26T21:53:02.864ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
78992025-08-26T21:53:02.865ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = alert_dispatcher resource = Database result = Ok(())
79002025-08-26T21:53:02.867ZDEBGnexus (BackgroundTasks): activation complete background_task = alert_dispatcher elapsed = 6.738552ms iteration = 8 status = {"dispatched":[],"errors":[],"glob_version":"183.0.0","globs_reprocessed":{},"no_receivers":[]}
79012025-08-26T21:53:02.868ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 15 reason = Timeout
79022025-08-26T21:53:02.868ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 15 reason = Timeout
79032025-08-26T21:53:02.868ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
79042025-08-26T21:53:02.868ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 12.141\u{b5}s iteration = 15 status = {"disabled":true,"errors":[],"sps":[]}
79052025-08-26T21:53:02.868ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 15 reason = Timeout
79062025-08-26T21:53:02.868ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 15 reason = Timeout
79072025-08-26T21:53:02.868ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
79082025-08-26T21:53:02.868ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
79092025-08-26T21:53:02.868ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 43 reason = Timeout
79102025-08-26T21:53:02.868ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
79112025-08-26T21:53:02.868ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
79122025-08-26T21:53:02.868ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 15 reason = Timeout
79132025-08-26T21:53:02.868ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
79142025-08-26T21:53:02.868ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 85 reason = Timeout
79152025-08-26T21:53:02.868ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
79162025-08-26T21:53:02.868ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 15 reason = Timeout
79172025-08-26T21:53:02.868ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
79182025-08-26T21:53:02.868ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 15 reason = Timeout
79192025-08-26T21:53:02.868ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
79202025-08-26T21:53:02.868ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
79212025-08-26T21:53:02.868ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 15 reason = Timeout
79222025-08-26T21:53:02.868ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
79232025-08-26T21:53:02.868ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 15 reason = Timeout
79242025-08-26T21:53:02.868ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
79252025-08-26T21:53:02.868ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 16 reason = Timeout
79262025-08-26T21:53:02.868ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
79272025-08-26T21:53:02.868ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 15 reason = Timeout
79282025-08-26T21:53:02.868ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
79292025-08-26T21:53:02.868ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 15 reason = Timeout
79302025-08-26T21:53:02.868ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
79312025-08-26T21:53:02.868ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 15 reason = Timeout
79322025-08-26T21:53:02.868ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
79332025-08-26T21:53:02.868ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 15 reason = Timeout
79342025-08-26T21:53:02.868ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
79352025-08-26T21:53:02.868ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 15 reason = Timeout
79362025-08-26T21:53:02.868ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
79372025-08-26T21:53:02.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
79382025-08-26T21:53:02.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
79392025-08-26T21:53:02.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
79402025-08-26T21:53:02.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
79412025-08-26T21:53:02.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
79422025-08-26T21:53:02.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
79432025-08-26T21:53:02.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
79442025-08-26T21:53:02.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
79452025-08-26T21:53:02.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
79462025-08-26T21:53:02.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
79472025-08-26T21:53:02.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
79482025-08-26T21:53:02.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
79492025-08-26T21:53:02.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
79502025-08-26T21:53:02.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
79512025-08-26T21:53:02.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
79522025-08-26T21:53:02.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
79532025-08-26T21:53:02.870ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
79542025-08-26T21:53:02.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
79552025-08-26T21:53:02.872ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 43 reason = Timeout
79562025-08-26T21:53:02.872ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
79572025-08-26T21:53:02.872ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
79582025-08-26T21:53:02.873ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
79592025-08-26T21:53:02.873ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
79602025-08-26T21:53:02.875ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
79612025-08-26T21:53:02.877ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
79622025-08-26T21:53:02.877ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
79632025-08-26T21:53:02.877ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
79642025-08-26T21:53:02.878ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
79652025-08-26T21:53:02.878ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
79662025-08-26T21:53:02.879ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
79672025-08-26T21:53:02.879ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
79682025-08-26T21:53:02.879ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
79692025-08-26T21:53:02.881ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
79702025-08-26T21:53:02.881ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
79712025-08-26T21:53:02.881ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
79722025-08-26T21:53:02.881ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
79732025-08-26T21:53:02.881ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
79742025-08-26T21:53:02.882ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
79752025-08-26T21:53:02.882ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
79762025-08-26T21:53:02.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
79772025-08-26T21:53:02.883ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1286, tv_nsec: 741916230 } })
79782025-08-26T21:53:02.883ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
79792025-08-26T21:53:02.883ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
79802025-08-26T21:53:02.883ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1205 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
79812025-08-26T21:53:02.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
79822025-08-26T21:53:02.883ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
79832025-08-26T21:53:02.884ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
79842025-08-26T21:53:02.884ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 15.463168ms iteration = 85 status = {"chicken_switches_updated":false}
79852025-08-26T21:53:02.885ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
79862025-08-26T21:53:02.886ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
79872025-08-26T21:53:02.887ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
79882025-08-26T21:53:02.887ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
79892025-08-26T21:53:02.887ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
79902025-08-26T21:53:02.888ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
79912025-08-26T21:53:02.888ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
79922025-08-26T21:53:02.888ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
79932025-08-26T21:53:02.888ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
79942025-08-26T21:53:02.889ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
79952025-08-26T21:53:02.890ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
79962025-08-26T21:53:02.890ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
79972025-08-26T21:53:02.890ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
79982025-08-26T21:53:02.890ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
79992025-08-26T21:53:02.890ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
80002025-08-26T21:53:02.891ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
80012025-08-26T21:53:02.891ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
80022025-08-26T21:53:02.891ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
80032025-08-26T21:53:02.891ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
80042025-08-26T21:53:02.891ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
80052025-08-26T21:53:02.892ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
80062025-08-26T21:53:02.892ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
80072025-08-26T21:53:02.892ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
80082025-08-26T21:53:02.893ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
80092025-08-26T21:53:02.900ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
80102025-08-26T21:53:02.901ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
80112025-08-26T21:53:02.901ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
80122025-08-26T21:53:02.901ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
80132025-08-26T21:53:02.902ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
80142025-08-26T21:53:02.902ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
80152025-08-26T21:53:02.903ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
80162025-08-26T21:53:02.903ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 8e720f02-dde8-4423-a6e3-91698373805a dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
80172025-08-26T21:53:02.903ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
80182025-08-26T21:53:02.904ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
80192025-08-26T21:53:02.905ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
80202025-08-26T21:53:02.905ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
80212025-08-26T21:53:02.906ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
80222025-08-26T21:53:02.908ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
80232025-08-26T21:53:02.914ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
80242025-08-26T21:53:02.914ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
80252025-08-26T21:53:02.914ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
80262025-08-26T21:53:02.916ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 8e720f02-dde8-4423-a6e3-91698373805a dns_zone_name = control-plane.oxide.internal found_so_far = 59 version = 1
80272025-08-26T21:53:02.916ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
80282025-08-26T21:53:02.916ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2025-08-26 21:46:04.065217 UTC dns_group = internal generation = 1
80292025-08-26T21:53:02.916ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 83.585234ms iteration = 8 status = {"generation":1}
80302025-08-26T21:53:02.920ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
80312025-08-26T21:53:02.920ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 48.514424ms iteration = 43 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
80322025-08-26T21:53:02.929ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
80332025-08-26T21:53:02.929ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
80342025-08-26T21:53:02.929ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
80352025-08-26T21:53:02.930ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
80362025-08-26T21:53:02.935ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
80372025-08-26T21:53:02.935ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
80382025-08-26T21:53:02.937ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
80392025-08-26T21:53:02.937ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:138 found = 0 not_ready_to_be_deleted = 0
80402025-08-26T21:53:02.937ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 77.101078ms iteration = 8 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
80412025-08-26T21:53:02.937ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
80422025-08-26T21:53:02.938ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
80432025-08-26T21:53:02.939ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
80442025-08-26T21:53:02.942ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
80452025-08-26T21:53:02.942ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
80462025-08-26T21:53:02.943ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
80472025-08-26T21:53:02.943ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 74.478509ms iteration = 15 status = {"errors":[],"garbage_collect_requested":[]}
80482025-08-26T21:53:02.945ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
80492025-08-26T21:53:02.947ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 78.641643ms iteration = 43 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
80502025-08-26T21:53:02.947ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 78.487029ms iteration = 15 status = {"errors":[],"finish_invoked_ok":[]}
80512025-08-26T21:53:02.948ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
80522025-08-26T21:53:02.948ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
80532025-08-26T21:53:02.948ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
80542025-08-26T21:53:02.948ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 80.167927ms iteration = 16 status = {"physical_disks_added":0}
80552025-08-26T21:53:02.950ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
80562025-08-26T21:53:02.950ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
80572025-08-26T21:53:02.950ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
80582025-08-26T21:53:02.951ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
80592025-08-26T21:53:02.951ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
80602025-08-26T21:53:02.951ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
80612025-08-26T21:53:02.953ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
80622025-08-26T21:53:02.955ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
80632025-08-26T21:53:02.955ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 86.93117ms iteration = 15 status = {"errors":[],"requests_created_ok":[]}
80642025-08-26T21:53:02.955ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
80652025-08-26T21:53:02.956ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
80662025-08-26T21:53:02.956ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
80672025-08-26T21:53:02.956ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
80682025-08-26T21:53:02.964ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
80692025-08-26T21:53:02.967ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
80702025-08-26T21:53:02.967ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
80712025-08-26T21:53:02.967ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
80722025-08-26T21:53:02.975ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-08-26T21:45:59.866217Z, time_modified: 2025-08-26T21:45:59.866217Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 9396aa55-de6f-4d6f-96db-d28e1c76fa9c, db_silo: Silo { identity: SiloIdentity { id: 9396aa55-de6f-4d6f-96db-d28e1c76fa9c, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-08-26T21:46:05.245521Z, time_modified: 2025-08-26T21:46:05.245521Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "e8529d720e4b0de099753dcdb373abb870af04baacd7c7dd850f5254858081af" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 9396aa55-de6f-4d6f-96db-d28e1c76fa9c, db_silo: Silo { identity: SiloIdentity { id: 9396aa55-de6f-4d6f-96db-d28e1c76fa9c, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-08-26T21:46:05.245521Z, time_modified: 2025-08-26T21:46:05.245521Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "e8529d720e4b0de099753dcdb373abb870af04baacd7c7dd850f5254858081af" }] }) }
80732025-08-26T21:53:02.975ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 142.532543ms iteration = 8 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"9396aa55-de6f-4d6f-96db-d28e1c76fa9c","tls_certs":["e8529d720e4b0de099753dcdb373abb870af04baacd7c7dd850f5254858081af"]}},"default_endpoint":{"silo_id":"9396aa55-de6f-4d6f-96db-d28e1c76fa9c","tls_certs":["e8529d720e4b0de099753dcdb373abb870af04baacd7c7dd850f5254858081af"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
80742025-08-26T21:53:02.976ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
80752025-08-26T21:53:02.989ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1213 zone_address = fd00:1122:3344:101::2
80762025-08-26T21:53:02.989ZDEBGnexus (MgsClient): client request background_task = nat_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
80772025-08-26T21:53:02.990ZDEBGnexus (MgsClient): client response background_task = nat_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "c7399800-d912-4c1a-8e44-c7bc9848ffdf", "content-length": "26", "date": "Tue, 26 Aug 2025 21:53:02 GMT"} })
80782025-08-26T21:53:02.990ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1216 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
80792025-08-26T21:53:02.992ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
80802025-08-26T21:53:02.992ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
80812025-08-26T21:53:02.992ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
80822025-08-26T21:53:02.992ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
80832025-08-26T21:53:02.992ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
80842025-08-26T21:53:02.994ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 125.407067ms iteration = 15 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
80852025-08-26T21:53:02.998ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1213 zone_address = fd00:1122:3344:101::2
80862025-08-26T21:53:02.998ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
80872025-08-26T21:53:02.998ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "6c4e18da-000b-4431-ad31-3ee66a9a1e71", "content-length": "26", "date": "Tue, 26 Aug 2025 21:53:02 GMT"} })
80882025-08-26T21:53:02.998ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1216 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
80892025-08-26T21:53:03.000ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
80902025-08-26T21:53:03.003ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
80912025-08-26T21:53:03.003ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
80922025-08-26T21:53:03.003ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
80932025-08-26T21:53:03.003ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 135.034424ms iteration = 15 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
80942025-08-26T21:53:03.003ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 135.30351ms iteration = 15 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
80952025-08-26T21:53:03.005ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
80962025-08-26T21:53:03.005ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
80972025-08-26T21:53:03.005ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
80982025-08-26T21:53:03.005ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
80992025-08-26T21:53:03.005ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
81002025-08-26T21:53:03.008ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:550 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
81012025-08-26T21:53:03.008ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 139.936355ms iteration = 15 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
81022025-08-26T21:53:03.010ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
81032025-08-26T21:53:03.010ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = dfb404ab-acb8-43de-afe8-b429af0c161d dns_zone_name = oxide.test found_so_far = 0 version = 2
81042025-08-26T21:53:03.010ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
81052025-08-26T21:53:03.011ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
81062025-08-26T21:53:03.012ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
81072025-08-26T21:53:03.013ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
81082025-08-26T21:53:03.014ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
81092025-08-26T21:53:03.016ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
81102025-08-26T21:53:03.016ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
81112025-08-26T21:53:03.016ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
81122025-08-26T21:53:03.018ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = dfb404ab-acb8-43de-afe8-b429af0c161d dns_zone_name = oxide.test found_so_far = 4 version = 2
81132025-08-26T21:53:03.018ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
81142025-08-26T21:53:03.018ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2025-08-26 21:46:05.296205 UTC dns_group = external generation = 2
81152025-08-26T21:53:03.018ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 185.322377ms iteration = 8 status = {"generation":2}
81162025-08-26T21:53:03.032ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
81172025-08-26T21:53:03.033ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
81182025-08-26T21:53:03.048ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:407
81192025-08-26T21:53:03.048ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 179.942256ms iteration = 15 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
81202025-08-26T21:53:03.049ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
81212025-08-26T21:53:03.052ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
81222025-08-26T21:53:03.052ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
81232025-08-26T21:53:03.052ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 184.08807ms iteration = 15 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
81242025-08-26T21:53:03.053ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: dd0c655a-95c2-48fc-b4c3-9733ed4b0faa, lookup_type: ById(dd0c655a-95c2-48fc-b4c3-9733ed4b0faa) } result = Ok(())
81252025-08-26T21:53:03.053ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
81262025-08-26T21:53:03.053ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
81272025-08-26T21:53:03.055ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 186.598037ms iteration = 15 status = 0
81282025-08-26T21:53:03.086ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
81292025-08-26T21:53:03.095ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1213 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 zone_address = fd00:1122:3344:101::2
81302025-08-26T21:53:03.095ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
81312025-08-26T21:53:03.096ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "d4d438d5-0240-4c03-be95-61d00f2bed34", "content-length": "26", "date": "Tue, 26 Aug 2025 21:53:02 GMT"} })
81322025-08-26T21:53:03.096ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1216 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
81332025-08-26T21:53:03.097ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
81342025-08-26T21:53:03.124ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
81352025-08-26T21:53:03.132ZINFOnexus (BackgroundTasks): instance reincarnation completed successfully background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:113 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0
81362025-08-26T21:53:03.132ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 272.297158ms iteration = 8 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
81372025-08-26T21:53:03.158ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
81382025-08-26T21:53:03.158ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
81392025-08-26T21:53:03.160ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
81402025-08-26T21:53:03.162ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
81412025-08-26T21:53:03.162ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
81422025-08-26T21:53:03.162ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
81432025-08-26T21:53:03.164ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
81442025-08-26T21:53:03.165ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
81452025-08-26T21:53:03.165ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
81462025-08-26T21:53:03.166ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
81472025-08-26T21:53:03.172ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 339.477773ms iteration = 15 status = {}
81482025-08-26T21:53:03.178ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
81492025-08-26T21:53:03.178ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
81502025-08-26T21:53:03.180ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 347.3672ms iteration = 15 status = null
81512025-08-26T21:53:03.236ZINFOnexus (BackgroundTasks): VPC route manager sled dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
81522025-08-26T21:53:03.236ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
81532025-08-26T21:53:03.236ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "21c75043-7f9a-482c-9327-0c1c969417e5", "content-length": "372", "date": "Tue, 26 Aug 2025 21:53:03 GMT"} })
81542025-08-26T21:53:03.236ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
81552025-08-26T21:53:03.237ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
81562025-08-26T21:53:03.251ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
81572025-08-26T21:53:03.251ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
81582025-08-26T21:53:03.251ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "81c33e31-aa78-4386-8ace-376649212184", "date": "Tue, 26 Aug 2025 21:53:03 GMT"} })
81592025-08-26T21:53:03.251ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
81602025-08-26T21:53:03.251ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
81612025-08-26T21:53:03.254ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
81622025-08-26T21:53:03.254ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
81632025-08-26T21:53:03.256ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
81642025-08-26T21:53:03.256ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
81652025-08-26T21:53:03.260ZINFOnexus (BackgroundTasks): VPC route manager sled dd0c655a-95c2-48fc-b4c3-9733ed4b0faa push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
81662025-08-26T21:53:03.260ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
81672025-08-26T21:53:03.260ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "0f9d3d95-522e-4e9e-a217-67208069e893", "date": "Tue, 26 Aug 2025 21:53:03 GMT"} })
81682025-08-26T21:53:03.262ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 393.782056ms iteration = 15 status = {}
81692025-08-26T21:53:03.264ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
81702025-08-26T21:53:03.265ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "cd2970be-303f-4425-8cfa-e020eadcf1d3", "date": "Tue, 26 Aug 2025 21:53:03 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
81712025-08-26T21:53:03.265ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
81722025-08-26T21:53:03.267ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "1dffa5c4-5d99-4352-b546-c04340128c2b", "date": "Tue, 26 Aug 2025 21:53:03 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
81732025-08-26T21:53:03.267ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
81742025-08-26T21:53:03.269ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "72bf4f35-0086-4b15-b87d-27e98b702d86", "date": "Tue, 26 Aug 2025 21:53:03 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
81752025-08-26T21:53:03.269ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
81762025-08-26T21:53:03.271ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
81772025-08-26T21:53:03.271ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 438.504844ms iteration = 10 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
81782025-08-26T21:53:03.286ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
81792025-08-26T21:53:03.287ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "6af464a0-2d5d-4b3b-8a02-da3b1df89d76", "date": "Tue, 26 Aug 2025 21:53:03 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
81802025-08-26T21:53:03.287ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
81812025-08-26T21:53:03.288ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "1afdb62f-cc9c-4a44-aa4b-06874aacca85", "date": "Tue, 26 Aug 2025 21:53:03 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
81822025-08-26T21:53:03.288ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
81832025-08-26T21:53:03.290ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
81842025-08-26T21:53:03.290ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 457.85335ms iteration = 10 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
81852025-08-26T21:53:03.292ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
81862025-08-26T21:53:03.293ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "404d59d4-c7c4-4f0c-b4f0-3a71783df95c", "content-length": "691", "date": "Tue, 26 Aug 2025 21:53:03 GMT"} })
81872025-08-26T21:53:03.293ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
81882025-08-26T21:53:03.293ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
81892025-08-26T21:53:03.293ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
81902025-08-26T21:53:03.294ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 426.065474ms iteration = 15 status = {}
81912025-08-26T21:53:03.337ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
81922025-08-26T21:53:03.338ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
81932025-08-26T21:53:03.340ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: d24c5c55-3a21-4a91-aeb4-45362e822df6, rack_id: e37a46f2-f789-4ca4-9841-f8c6dd705964, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(88d84f37-a4c8-4abc-af1f-27987b0d39d9) } rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0
81942025-08-26T21:53:03.340ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
81952025-08-26T21:53:03.340ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
81962025-08-26T21:53:03.352ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1832 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
81972025-08-26T21:53:03.352ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1859 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 88d84f37-a4c8-4abc-af1f-27987b0d39d9
81982025-08-26T21:53:03.353ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
81992025-08-26T21:53:03.353ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
82002025-08-26T21:53:03.353ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
82012025-08-26T21:53:03.353ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
82022025-08-26T21:53:03.353ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
82032025-08-26T21:53:03.353ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2006 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
82042025-08-26T21:53:03.353ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:408 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
82052025-08-26T21:53:03.353ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2042 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
82062025-08-26T21:53:03.353ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
82072025-08-26T21:53:03.353ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
82082025-08-26T21:53:03.353ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
82092025-08-26T21:53:03.356ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0
82102025-08-26T21:53:03.356ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
82112025-08-26T21:53:03.460ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "9c9c45a1-82b4-4f79-b0a7-d5691c450bb3", "date": "Tue, 26 Aug 2025 21:53:03 GMT"} })
82122025-08-26T21:53:03.460ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch = Switch0
82132025-08-26T21:53:03.460ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
82142025-08-26T21:53:03.460ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "7f4717dd-9f78-40d8-a5d8-fe4483e81bb5", "content-length": "682", "date": "Tue, 26 Aug 2025 21:53:03 GMT"} })
82152025-08-26T21:53:03.461ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
82162025-08-26T21:53:03.461ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
82172025-08-26T21:53:03.464ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
82182025-08-26T21:53:03.464ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
82192025-08-26T21:53:03.467ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
82202025-08-26T21:53:03.467ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
82212025-08-26T21:53:03.470ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
82222025-08-26T21:53:03.470ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
82232025-08-26T21:53:03.478ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 609.574011ms iteration = 15 status = {}
82242025-08-26T21:53:05.352ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 334 local_addr = [fd00:1122:3344:101::c]:42016 method = GET remote_addr = [fd00:1122:3344:101::d]:37550 req_id = 3e859b68-99b2-468a-9319-81c6a43495fd response_code = 200 uri = /5d0959e5-47a6-4bf3-a52b-57950b9d900d
82252025-08-26T21:53:07.869ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 86 reason = Timeout
82262025-08-26T21:53:07.869ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
82272025-08-26T21:53:07.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
82282025-08-26T21:53:07.871ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
82292025-08-26T21:53:07.874ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
82302025-08-26T21:53:07.874ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
82312025-08-26T21:53:07.874ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
82322025-08-26T21:53:07.876ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
82332025-08-26T21:53:07.876ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.458718ms iteration = 86 status = {"chicken_switches_updated":false}
82342025-08-26T21:53:12.868ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 44 reason = Timeout
82352025-08-26T21:53:12.868ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
82362025-08-26T21:53:12.868ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 87 reason = Timeout
82372025-08-26T21:53:12.868ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
82382025-08-26T21:53:12.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
82392025-08-26T21:53:12.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
82402025-08-26T21:53:12.871ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
82412025-08-26T21:53:12.871ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 44 reason = Timeout
82422025-08-26T21:53:12.871ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
82432025-08-26T21:53:12.871ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
82442025-08-26T21:53:12.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
82452025-08-26T21:53:12.872ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
82462025-08-26T21:53:12.873ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
82472025-08-26T21:53:12.874ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
82482025-08-26T21:53:12.874ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
82492025-08-26T21:53:12.874ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
82502025-08-26T21:53:12.876ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
82512025-08-26T21:53:12.876ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
82522025-08-26T21:53:12.876ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
82532025-08-26T21:53:12.876ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
82542025-08-26T21:53:12.876ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.884107ms iteration = 87 status = {"chicken_switches_updated":false}
82552025-08-26T21:53:12.878ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
82562025-08-26T21:53:12.878ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
82572025-08-26T21:53:12.880ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
82582025-08-26T21:53:12.882ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
82592025-08-26T21:53:12.882ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
82602025-08-26T21:53:12.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
82612025-08-26T21:53:12.900ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
82622025-08-26T21:53:12.900ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 28.719697ms iteration = 44 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
82632025-08-26T21:53:12.909ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 41.171069ms iteration = 44 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
82642025-08-26T21:53:15.345ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:1025 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::d]:61061
82652025-08-26T21:53:15.345ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:61061 req_id = f940715b-4952-4a2d-9bb3-6c02d6a239a6 roles = RoleSet { roles: {} } uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?sort_by=id_ascending
82662025-08-26T21:53:15.345ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:61061 req_id = f940715b-4952-4a2d-9bb3-6c02d6a239a6 resource = Database result = Ok(()) uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?sort_by=id_ascending
82672025-08-26T21:53:15.348ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 3455 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:61061 req_id = f940715b-4952-4a2d-9bb3-6c02d6a239a6 response_code = 200 uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?sort_by=id_ascending
82682025-08-26T21:53:15.348ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:61061 req_id = 6047508e-bb10-41f3-8d8d-57d87e22cc6a roles = RoleSet { roles: {} } uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMzhjYTg4YS1lZGE1LTQwZGEtOGE4MS1hN2UzMzdhMmM4ZmIifX0%3D
82692025-08-26T21:53:15.349ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:61061 req_id = 6047508e-bb10-41f3-8d8d-57d87e22cc6a resource = Database result = Ok(()) uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMzhjYTg4YS1lZGE1LTQwZGEtOGE4MS1hN2UzMzdhMmM4ZmIifX0%3D
82702025-08-26T21:53:15.351ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 2460 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:61061 req_id = 6047508e-bb10-41f3-8d8d-57d87e22cc6a response_code = 200 uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMzhjYTg4YS1lZGE1LTQwZGEtOGE4MS1hN2UzMzdhMmM4ZmIifX0%3D
82712025-08-26T21:53:15.351ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 270 local_addr = [fd00:1122:3344:101::c]:42016 method = GET remote_addr = [fd00:1122:3344:101::d]:37550 req_id = 9a9f62b1-e47c-4eed-8ee6-ad6e9cb5a68b response_code = 200 uri = /5d0959e5-47a6-4bf3-a52b-57950b9d900d
82722025-08-26T21:53:17.869ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 88 reason = Timeout
82732025-08-26T21:53:17.869ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
82742025-08-26T21:53:17.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
82752025-08-26T21:53:17.871ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
82762025-08-26T21:53:17.874ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
82772025-08-26T21:53:17.874ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
82782025-08-26T21:53:17.874ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
82792025-08-26T21:53:17.877ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
82802025-08-26T21:53:17.877ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 8.195565ms iteration = 88 status = {"chicken_switches_updated":false}
82812025-08-26T21:53:22.870ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 45 reason = Timeout
82822025-08-26T21:53:22.870ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 89 reason = Timeout
82832025-08-26T21:53:22.870ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
82842025-08-26T21:53:22.870ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
82852025-08-26T21:53:22.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
82862025-08-26T21:53:22.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
82872025-08-26T21:53:22.872ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
82882025-08-26T21:53:22.872ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
82892025-08-26T21:53:22.872ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 45 reason = Timeout
82902025-08-26T21:53:22.872ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
82912025-08-26T21:53:22.872ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
82922025-08-26T21:53:22.872ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
82932025-08-26T21:53:22.874ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
82942025-08-26T21:53:22.874ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.353958ms iteration = 45 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
82952025-08-26T21:53:22.875ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
82962025-08-26T21:53:22.875ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
82972025-08-26T21:53:22.875ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
82982025-08-26T21:53:22.877ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
82992025-08-26T21:53:22.877ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.457538ms iteration = 89 status = {"chicken_switches_updated":false}
83002025-08-26T21:53:22.877ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
83012025-08-26T21:53:22.877ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
83022025-08-26T21:53:22.877ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
83032025-08-26T21:53:22.879ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
83042025-08-26T21:53:22.879ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
83052025-08-26T21:53:22.881ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
83062025-08-26T21:53:22.883ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
83072025-08-26T21:53:22.883ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
83082025-08-26T21:53:22.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
83092025-08-26T21:53:22.901ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
83102025-08-26T21:53:22.901ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 28.8254ms iteration = 45 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
83112025-08-26T21:53:25.352ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 282 local_addr = [fd00:1122:3344:101::c]:42016 method = GET remote_addr = [fd00:1122:3344:101::d]:37550 req_id = 67e9ef0f-bd40-4f59-a922-267b1e592879 response_code = 200 uri = /5d0959e5-47a6-4bf3-a52b-57950b9d900d
83122025-08-26T21:53:27.869ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 90 reason = Timeout
83132025-08-26T21:53:27.869ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
83142025-08-26T21:53:27.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
83152025-08-26T21:53:27.872ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
83162025-08-26T21:53:27.874ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
83172025-08-26T21:53:27.874ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
83182025-08-26T21:53:27.875ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
83192025-08-26T21:53:27.877ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
83202025-08-26T21:53:27.877ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.630442ms iteration = 90 status = {"chicken_switches_updated":false}
83212025-08-26T21:53:32.833ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 16 reason = Timeout
83222025-08-26T21:53:32.833ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 16 reason = Timeout
83232025-08-26T21:53:32.833ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 16 reason = Timeout
83242025-08-26T21:53:32.833ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
83252025-08-26T21:53:32.833ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
83262025-08-26T21:53:32.833ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
83272025-08-26T21:53:32.833ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
83282025-08-26T21:53:32.836ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1316, tv_nsec: 694475730 } })
83292025-08-26T21:53:32.836ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.072269ms iteration = 16 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
83302025-08-26T21:53:32.836ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1316, tv_nsec: 694616323 } })
83312025-08-26T21:53:32.836ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1205
83322025-08-26T21:53:32.836ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1205
83332025-08-26T21:53:32.869ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 16 reason = Timeout
83342025-08-26T21:53:32.869ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 16 reason = Timeout
83352025-08-26T21:53:32.869ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 11.24\u{b5}s iteration = 16 status = {"disabled":true,"errors":[],"sps":[]}
83362025-08-26T21:53:32.869ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 16 reason = Timeout
83372025-08-26T21:53:32.869ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
83382025-08-26T21:53:32.869ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
83392025-08-26T21:53:32.869ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 16 reason = Timeout
83402025-08-26T21:53:32.869ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
83412025-08-26T21:53:32.869ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 16 reason = Timeout
83422025-08-26T21:53:32.869ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
83432025-08-26T21:53:32.869ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 46 reason = Timeout
83442025-08-26T21:53:32.869ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 16 reason = Timeout
83452025-08-26T21:53:32.869ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
83462025-08-26T21:53:32.870ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
83472025-08-26T21:53:32.870ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 91 reason = Timeout
83482025-08-26T21:53:32.870ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
83492025-08-26T21:53:32.870ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 16 reason = Timeout
83502025-08-26T21:53:32.870ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
83512025-08-26T21:53:32.870ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 16 reason = Timeout
83522025-08-26T21:53:32.870ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
83532025-08-26T21:53:32.870ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 17 reason = Timeout
83542025-08-26T21:53:32.870ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
83552025-08-26T21:53:32.870ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 16 reason = Timeout
83562025-08-26T21:53:32.870ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
83572025-08-26T21:53:32.870ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 16 reason = Timeout
83582025-08-26T21:53:32.870ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
83592025-08-26T21:53:32.870ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
83602025-08-26T21:53:32.870ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 16 reason = Timeout
83612025-08-26T21:53:32.870ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
83622025-08-26T21:53:32.870ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 16 reason = Timeout
83632025-08-26T21:53:32.870ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 16 reason = Timeout
83642025-08-26T21:53:32.870ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
83652025-08-26T21:53:32.870ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
83662025-08-26T21:53:32.870ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 16 reason = Timeout
83672025-08-26T21:53:32.870ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
83682025-08-26T21:53:32.870ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
83692025-08-26T21:53:32.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
83702025-08-26T21:53:32.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
83712025-08-26T21:53:32.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
83722025-08-26T21:53:32.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
83732025-08-26T21:53:32.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
83742025-08-26T21:53:32.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
83752025-08-26T21:53:32.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
83762025-08-26T21:53:32.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
83772025-08-26T21:53:32.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
83782025-08-26T21:53:32.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
83792025-08-26T21:53:32.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
83802025-08-26T21:53:32.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
83812025-08-26T21:53:32.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
83822025-08-26T21:53:32.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
83832025-08-26T21:53:32.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
83842025-08-26T21:53:32.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
83852025-08-26T21:53:32.872ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 46 reason = Timeout
83862025-08-26T21:53:32.872ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
83872025-08-26T21:53:32.872ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 2.63574ms iteration = 16 status = {"errors":[],"finish_invoked_ok":[]}
83882025-08-26T21:53:32.872ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
83892025-08-26T21:53:32.872ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
83902025-08-26T21:53:32.872ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 2.723182ms iteration = 16 status = {"errors":[],"garbage_collect_requested":[]}
83912025-08-26T21:53:32.872ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
83922025-08-26T21:53:32.872ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
83932025-08-26T21:53:32.873ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
83942025-08-26T21:53:32.874ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1213 zone_address = fd00:1122:3344:101::2
83952025-08-26T21:53:32.874ZDEBGnexus (MgsClient): client request background_task = nat_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
83962025-08-26T21:53:32.874ZDEBGnexus (MgsClient): client response background_task = nat_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "cc6c0425-9bd9-4e8b-9109-01bdf12c8764", "content-length": "26", "date": "Tue, 26 Aug 2025 21:53:32 GMT"} })
83972025-08-26T21:53:32.874ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1216 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
83982025-08-26T21:53:32.875ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 6.022145ms iteration = 16 status = {"errors":[],"requests_created_ok":[]}
83992025-08-26T21:53:32.876ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
84002025-08-26T21:53:32.876ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
84012025-08-26T21:53:32.876ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
84022025-08-26T21:53:32.876ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1213 zone_address = fd00:1122:3344:101::2
84032025-08-26T21:53:32.876ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
84042025-08-26T21:53:32.876ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
84052025-08-26T21:53:32.876ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "348b0be2-70ae-40dd-80c7-705164f73616", "content-length": "26", "date": "Tue, 26 Aug 2025 21:53:32 GMT"} })
84062025-08-26T21:53:32.876ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1216 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
84072025-08-26T21:53:32.877ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
84082025-08-26T21:53:32.877ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
84092025-08-26T21:53:32.877ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
84102025-08-26T21:53:32.877ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
84112025-08-26T21:53:32.877ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
84122025-08-26T21:53:32.877ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
84132025-08-26T21:53:32.877ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
84142025-08-26T21:53:32.877ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
84152025-08-26T21:53:32.877ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
84162025-08-26T21:53:32.878ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
84172025-08-26T21:53:32.879ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
84182025-08-26T21:53:32.879ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
84192025-08-26T21:53:32.883ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
84202025-08-26T21:53:32.884ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
84212025-08-26T21:53:32.884ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 14.254591ms iteration = 91 status = {"chicken_switches_updated":false}
84222025-08-26T21:53:32.884ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 14.749803ms iteration = 16 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
84232025-08-26T21:53:32.886ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
84242025-08-26T21:53:32.886ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
84252025-08-26T21:53:32.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
84262025-08-26T21:53:32.891ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
84272025-08-26T21:53:32.892ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
84282025-08-26T21:53:32.903ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
84292025-08-26T21:53:32.903ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
84302025-08-26T21:53:32.905ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
84312025-08-26T21:53:32.908ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
84322025-08-26T21:53:32.908ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
84332025-08-26T21:53:32.909ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
84342025-08-26T21:53:32.911ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
84352025-08-26T21:53:32.913ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
84362025-08-26T21:53:32.913ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
84372025-08-26T21:53:32.913ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
84382025-08-26T21:53:32.917ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
84392025-08-26T21:53:32.921ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
84402025-08-26T21:53:32.921ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
84412025-08-26T21:53:32.921ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
84422025-08-26T21:53:32.921ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
84432025-08-26T21:53:32.921ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
84442025-08-26T21:53:32.923ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 53.133408ms iteration = 46 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
84452025-08-26T21:53:32.923ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
84462025-08-26T21:53:32.924ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
84472025-08-26T21:53:32.924ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
84482025-08-26T21:53:32.924ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
84492025-08-26T21:53:32.924ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
84502025-08-26T21:53:32.924ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
84512025-08-26T21:53:32.926ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
84522025-08-26T21:53:32.927ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
84532025-08-26T21:53:32.927ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
84542025-08-26T21:53:32.927ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
84552025-08-26T21:53:32.927ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
84562025-08-26T21:53:32.927ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
84572025-08-26T21:53:32.927ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
84582025-08-26T21:53:32.927ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
84592025-08-26T21:53:32.928ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
84602025-08-26T21:53:32.928ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
84612025-08-26T21:53:32.929ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
84622025-08-26T21:53:32.929ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
84632025-08-26T21:53:32.929ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
84642025-08-26T21:53:32.929ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
84652025-08-26T21:53:32.929ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
84662025-08-26T21:53:32.929ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
84672025-08-26T21:53:32.930ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
84682025-08-26T21:53:32.930ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
84692025-08-26T21:53:32.930ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
84702025-08-26T21:53:32.930ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
84712025-08-26T21:53:32.931ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
84722025-08-26T21:53:32.931ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
84732025-08-26T21:53:32.931ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
84742025-08-26T21:53:32.931ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
84752025-08-26T21:53:32.931ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
84762025-08-26T21:53:32.932ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
84772025-08-26T21:53:32.933ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
84782025-08-26T21:53:32.933ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
84792025-08-26T21:53:32.933ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
84802025-08-26T21:53:32.933ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
84812025-08-26T21:53:32.935ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
84822025-08-26T21:53:32.935ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
84832025-08-26T21:53:32.935ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
84842025-08-26T21:53:32.935ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
84852025-08-26T21:53:32.936ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
84862025-08-26T21:53:32.936ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
84872025-08-26T21:53:32.937ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:550 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
84882025-08-26T21:53:32.937ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 68.059584ms iteration = 16 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
84892025-08-26T21:53:32.943ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1316, tv_nsec: 801435941 } })
84902025-08-26T21:53:32.943ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1205 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
84912025-08-26T21:53:32.951ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
84922025-08-26T21:53:32.951ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
84932025-08-26T21:53:32.953ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 119.988105ms iteration = 16 status = null
84942025-08-26T21:53:32.956ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
84952025-08-26T21:53:32.956ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 83.920422ms iteration = 46 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
84962025-08-26T21:53:32.964ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 94.03413ms iteration = 16 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
84972025-08-26T21:53:32.968ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 135.526675ms iteration = 16 status = {}
84982025-08-26T21:53:32.970ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
84992025-08-26T21:53:32.970ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
85002025-08-26T21:53:32.970ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 99.974424ms iteration = 16 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
85012025-08-26T21:53:32.977ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
85022025-08-26T21:53:32.977ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
85032025-08-26T21:53:32.979ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
85042025-08-26T21:53:32.980ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
85052025-08-26T21:53:32.980ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
85062025-08-26T21:53:32.981ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 110.979322ms iteration = 16 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
85072025-08-26T21:53:32.981ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
85082025-08-26T21:53:32.982ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
85092025-08-26T21:53:32.983ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
85102025-08-26T21:53:32.983ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
85112025-08-26T21:53:32.984ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: dd0c655a-95c2-48fc-b4c3-9733ed4b0faa, lookup_type: ById(dd0c655a-95c2-48fc-b4c3-9733ed4b0faa) } result = Ok(())
85122025-08-26T21:53:32.984ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
85132025-08-26T21:53:32.985ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
85142025-08-26T21:53:32.987ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 117.008848ms iteration = 16 status = 0
85152025-08-26T21:53:32.988ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
85162025-08-26T21:53:32.989ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "57b0cc94-9f52-4fad-97c7-aae8db43ebd5", "content-length": "691", "date": "Tue, 26 Aug 2025 21:53:32 GMT"} })
85172025-08-26T21:53:32.989ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
85182025-08-26T21:53:32.989ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
85192025-08-26T21:53:32.989ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
85202025-08-26T21:53:32.991ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 121.337356ms iteration = 16 status = {}
85212025-08-26T21:53:33.012ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1213 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 zone_address = fd00:1122:3344:101::2
85222025-08-26T21:53:33.012ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
85232025-08-26T21:53:33.012ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "a7292223-680c-4276-8a6d-ef8a2e45e9ce", "content-length": "26", "date": "Tue, 26 Aug 2025 21:53:32 GMT"} })
85242025-08-26T21:53:33.012ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1216 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
85252025-08-26T21:53:33.013ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
85262025-08-26T21:53:33.027ZINFOnexus (BackgroundTasks): VPC route manager sled dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
85272025-08-26T21:53:33.027ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
85282025-08-26T21:53:33.028ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "2aab7552-615b-446b-bb94-c6c56943b873", "content-length": "372", "date": "Tue, 26 Aug 2025 21:53:32 GMT"} })
85292025-08-26T21:53:33.028ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
85302025-08-26T21:53:33.028ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
85312025-08-26T21:53:33.030ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:407
85322025-08-26T21:53:33.030ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 160.423487ms iteration = 16 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
85332025-08-26T21:53:33.065ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 195.061527ms iteration = 17 status = {"physical_disks_added":0}
85342025-08-26T21:53:33.078ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
85352025-08-26T21:53:33.078ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
85362025-08-26T21:53:33.078ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "3c6df48c-dd13-4940-8e4b-17522ecba7cb", "date": "Tue, 26 Aug 2025 21:53:32 GMT"} })
85372025-08-26T21:53:33.078ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
85382025-08-26T21:53:33.078ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
85392025-08-26T21:53:33.081ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
85402025-08-26T21:53:33.082ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
85412025-08-26T21:53:33.085ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
85422025-08-26T21:53:33.085ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
85432025-08-26T21:53:33.088ZINFOnexus (BackgroundTasks): VPC route manager sled dd0c655a-95c2-48fc-b4c3-9733ed4b0faa push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
85442025-08-26T21:53:33.088ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
85452025-08-26T21:53:33.089ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "d873e1e6-3e24-4ef4-a1cf-7443f75447cd", "date": "Tue, 26 Aug 2025 21:53:32 GMT"} })
85462025-08-26T21:53:33.090ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 220.274076ms iteration = 16 status = {}
85472025-08-26T21:53:33.113ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
85482025-08-26T21:53:33.114ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
85492025-08-26T21:53:33.116ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: d24c5c55-3a21-4a91-aeb4-45362e822df6, rack_id: e37a46f2-f789-4ca4-9841-f8c6dd705964, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(88d84f37-a4c8-4abc-af1f-27987b0d39d9) } rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0
85502025-08-26T21:53:33.116ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
85512025-08-26T21:53:33.116ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
85522025-08-26T21:53:33.128ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1832 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
85532025-08-26T21:53:33.129ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1859 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 88d84f37-a4c8-4abc-af1f-27987b0d39d9
85542025-08-26T21:53:33.129ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
85552025-08-26T21:53:33.129ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
85562025-08-26T21:53:33.129ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
85572025-08-26T21:53:33.129ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
85582025-08-26T21:53:33.129ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
85592025-08-26T21:53:33.129ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2006 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
85602025-08-26T21:53:33.129ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:408 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
85612025-08-26T21:53:33.129ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2042 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
85622025-08-26T21:53:33.129ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
85632025-08-26T21:53:33.129ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
85642025-08-26T21:53:33.130ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
85652025-08-26T21:53:33.132ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0
85662025-08-26T21:53:33.132ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
85672025-08-26T21:53:33.253ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "335c8106-6b40-40e9-b0c9-2aeb5ffd6c2c", "date": "Tue, 26 Aug 2025 21:53:32 GMT"} })
85682025-08-26T21:53:33.253ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch = Switch0
85692025-08-26T21:53:33.253ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
85702025-08-26T21:53:33.254ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "79485de4-ce79-4ec7-a5a5-4147c4de1635", "content-length": "682", "date": "Tue, 26 Aug 2025 21:53:33 GMT"} })
85712025-08-26T21:53:33.254ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
85722025-08-26T21:53:33.254ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
85732025-08-26T21:53:33.258ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
85742025-08-26T21:53:33.258ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
85752025-08-26T21:53:33.261ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
85762025-08-26T21:53:33.261ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
85772025-08-26T21:53:33.263ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
85782025-08-26T21:53:33.264ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
85792025-08-26T21:53:33.270ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 400.486268ms iteration = 16 status = {}
85802025-08-26T21:53:35.352ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 291 local_addr = [fd00:1122:3344:101::c]:42016 method = GET remote_addr = [fd00:1122:3344:101::d]:37550 req_id = 09800e83-dd1e-48e5-aa4c-9a40cefadf4d response_code = 200 uri = /5d0959e5-47a6-4bf3-a52b-57950b9d900d
85812025-08-26T21:53:37.869ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 92 reason = Timeout
85822025-08-26T21:53:37.869ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
85832025-08-26T21:53:37.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
85842025-08-26T21:53:37.871ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
85852025-08-26T21:53:37.874ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
85862025-08-26T21:53:37.874ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
85872025-08-26T21:53:37.874ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
85882025-08-26T21:53:37.876ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
85892025-08-26T21:53:37.876ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 6.555307ms iteration = 92 status = {"chicken_switches_updated":false}
85902025-08-26T21:53:42.869ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 47 reason = Timeout
85912025-08-26T21:53:42.869ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
85922025-08-26T21:53:42.869ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 93 reason = Timeout
85932025-08-26T21:53:42.869ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
85942025-08-26T21:53:42.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
85952025-08-26T21:53:42.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
85962025-08-26T21:53:42.872ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 47 reason = Timeout
85972025-08-26T21:53:42.872ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
85982025-08-26T21:53:42.873ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
85992025-08-26T21:53:42.874ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
86002025-08-26T21:53:42.877ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
86012025-08-26T21:53:42.877ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
86022025-08-26T21:53:42.877ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
86032025-08-26T21:53:42.879ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
86042025-08-26T21:53:42.879ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
86052025-08-26T21:53:42.912ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
86062025-08-26T21:53:42.912ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
86072025-08-26T21:53:42.914ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 45.007055ms iteration = 47 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
86082025-08-26T21:53:42.919ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
86092025-08-26T21:53:42.923ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
86102025-08-26T21:53:42.923ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
86112025-08-26T21:53:42.923ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
86122025-08-26T21:53:42.925ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
86132025-08-26T21:53:42.925ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 56.125525ms iteration = 93 status = {"chicken_switches_updated":false}
86142025-08-26T21:53:42.927ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
86152025-08-26T21:53:42.930ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
86162025-08-26T21:53:42.930ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
86172025-08-26T21:53:42.930ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
86182025-08-26T21:53:42.947ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
86192025-08-26T21:53:42.947ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 75.118223ms iteration = 47 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
86202025-08-26T21:53:45.345ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:61061 req_id = dbdbeef2-1b18-41ae-aae4-c9968e3372c8 roles = RoleSet { roles: {} } uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?sort_by=id_ascending
86212025-08-26T21:53:45.345ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:61061 req_id = dbdbeef2-1b18-41ae-aae4-c9968e3372c8 resource = Database result = Ok(()) uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?sort_by=id_ascending
86222025-08-26T21:53:45.347ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 2792 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:61061 req_id = dbdbeef2-1b18-41ae-aae4-c9968e3372c8 response_code = 200 uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?sort_by=id_ascending
86232025-08-26T21:53:45.348ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:61061 req_id = 4fa81804-6af3-496a-88da-fe805bb792fd roles = RoleSet { roles: {} } uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMzhjYTg4YS1lZGE1LTQwZGEtOGE4MS1hN2UzMzdhMmM4ZmIifX0%3D
86242025-08-26T21:53:45.348ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:61061 req_id = 4fa81804-6af3-496a-88da-fe805bb792fd resource = Database result = Ok(()) uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMzhjYTg4YS1lZGE1LTQwZGEtOGE4MS1hN2UzMzdhMmM4ZmIifX0%3D
86252025-08-26T21:53:45.350ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 1904 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:61061 req_id = 4fa81804-6af3-496a-88da-fe805bb792fd response_code = 200 uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMzhjYTg4YS1lZGE1LTQwZGEtOGE4MS1hN2UzMzdhMmM4ZmIifX0%3D
86262025-08-26T21:53:45.352ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 300 local_addr = [fd00:1122:3344:101::c]:42016 method = GET remote_addr = [fd00:1122:3344:101::d]:37550 req_id = 5876f22f-906d-459b-b0ae-5dd851631cce response_code = 200 uri = /5d0959e5-47a6-4bf3-a52b-57950b9d900d
86272025-08-26T21:53:47.869ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 94 reason = Timeout
86282025-08-26T21:53:47.870ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
86292025-08-26T21:53:47.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
86302025-08-26T21:53:47.872ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
86312025-08-26T21:53:47.875ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
86322025-08-26T21:53:47.875ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
86332025-08-26T21:53:47.875ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
86342025-08-26T21:53:47.877ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
86352025-08-26T21:53:47.877ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.383696ms iteration = 94 status = {"chicken_switches_updated":false}
86362025-08-26T21:53:52.869ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 48 reason = Timeout
86372025-08-26T21:53:52.869ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
86382025-08-26T21:53:52.869ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 95 reason = Timeout
86392025-08-26T21:53:52.869ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
86402025-08-26T21:53:52.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
86412025-08-26T21:53:52.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
86422025-08-26T21:53:52.871ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
86432025-08-26T21:53:52.871ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 48 reason = Timeout
86442025-08-26T21:53:52.871ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
86452025-08-26T21:53:52.872ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
86462025-08-26T21:53:52.873ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
86472025-08-26T21:53:52.875ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
86482025-08-26T21:53:52.875ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
86492025-08-26T21:53:52.875ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
86502025-08-26T21:53:52.876ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
86512025-08-26T21:53:52.876ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.233243ms iteration = 95 status = {"chicken_switches_updated":false}
86522025-08-26T21:53:52.877ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
86532025-08-26T21:53:52.877ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
86542025-08-26T21:53:52.877ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
86552025-08-26T21:53:52.879ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
86562025-08-26T21:53:52.879ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
86572025-08-26T21:53:52.881ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
86582025-08-26T21:53:52.884ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
86592025-08-26T21:53:52.884ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
86602025-08-26T21:53:52.884ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
86612025-08-26T21:53:52.900ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
86622025-08-26T21:53:52.900ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 28.905742ms iteration = 48 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
86632025-08-26T21:53:52.904ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
86642025-08-26T21:53:52.904ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
86652025-08-26T21:53:52.940ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 70.94821ms iteration = 48 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
86662025-08-26T21:53:55.353ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 287 local_addr = [fd00:1122:3344:101::c]:42016 method = GET remote_addr = [fd00:1122:3344:101::d]:37550 req_id = b46305f3-a26b-4c72-9f67-392886186388 response_code = 200 uri = /5d0959e5-47a6-4bf3-a52b-57950b9d900d
86672025-08-26T21:53:57.869ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 96 reason = Timeout
86682025-08-26T21:53:57.869ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
86692025-08-26T21:53:57.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
86702025-08-26T21:53:57.870ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
86712025-08-26T21:53:57.873ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
86722025-08-26T21:53:57.873ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
86732025-08-26T21:53:57.873ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
86742025-08-26T21:53:57.875ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
86752025-08-26T21:53:57.875ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 6.815133ms iteration = 96 status = {"chicken_switches_updated":false}
86762025-08-26T21:54:02.833ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 9 reason = Timeout
86772025-08-26T21:54:02.833ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 9 reason = Timeout
86782025-08-26T21:54:02.833ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
86792025-08-26T21:54:02.833ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 17 reason = Timeout
86802025-08-26T21:54:02.833ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
86812025-08-26T21:54:02.833ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 9 reason = Timeout
86822025-08-26T21:54:02.833ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-08-26 21:44:02.833330131 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
86832025-08-26T21:54:02.833ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
86842025-08-26T21:54:02.833ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 17 reason = Timeout
86852025-08-26T21:54:02.833ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 11 reason = Timeout
86862025-08-26T21:54:02.833ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
86872025-08-26T21:54:02.833ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 11 reason = Timeout
86882025-08-26T21:54:02.833ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 9 reason = Timeout
86892025-08-26T21:54:02.833ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
86902025-08-26T21:54:02.833ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
86912025-08-26T21:54:02.833ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 9 reason = Timeout
86922025-08-26T21:54:02.833ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 17 reason = Timeout
86932025-08-26T21:54:02.833ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
86942025-08-26T21:54:02.833ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
86952025-08-26T21:54:02.833ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 9 reason = Timeout
86962025-08-26T21:54:02.833ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
86972025-08-26T21:54:02.834ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("60abf731-31e1-4111-a3e2-65da89eaed87.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("84c849eb-0949-41d9-ad02-9a5509144564.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("84c849eb-0949-41d9-ad02-9a5509144564.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::8)) }], valid_until: Instant { tv_sec: 1346, tv_nsec: 692379093 } })
86982025-08-26T21:54:02.834ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
86992025-08-26T21:54:02.834ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
87002025-08-26T21:54:02.834ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
87012025-08-26T21:54:02.834ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
87022025-08-26T21:54:02.834ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
87032025-08-26T21:54:02.834ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("852fd1a3-de9a-47e8-a667-5e8dab880ad3.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("c68f1501-294c-4874-92ca-8fc0546a5870.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("d741a72f-8745-4723-9e80-e1e72987266b.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("d741a72f-8745-4723-9e80-e1e72987266b.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:3::1)) }], valid_until: Instant { tv_sec: 1346, tv_nsec: 692510686 } })
87042025-08-26T21:54:02.834ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
87052025-08-26T21:54:02.834ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.238418ms iteration = 9 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
87062025-08-26T21:54:02.834ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
87072025-08-26T21:54:02.834ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } dns_group = internal
87082025-08-26T21:54:02.834ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.170896ms iteration = 9 status = {"addresses":["[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:3::1]:5353"]}
87092025-08-26T21:54:02.834ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
87102025-08-26T21:54:02.834ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
87112025-08-26T21:54:02.835ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
87122025-08-26T21:54:02.836ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 10 reason = Timeout
87132025-08-26T21:54:02.836ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:91 reason = blueprint disabled target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8
87142025-08-26T21:54:02.836ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 12.93\u{b5}s iteration = 10 status = {"enabled":false,"target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8"}
87152025-08-26T21:54:02.836ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-08-26 21:44:02.833330131 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
87162025-08-26T21:54:02.836ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 2.813233ms iteration = 9 status = {"expiration":"2025-08-26T21:44:02.833330131Z","pruned":[]}
87172025-08-26T21:54:02.836ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1346, tv_nsec: 694943150 } })
87182025-08-26T21:54:02.836ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
87192025-08-26T21:54:02.836ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1205
87202025-08-26T21:54:02.836ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1346, tv_nsec: 695346670 } })
87212025-08-26T21:54:02.837ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1205
87222025-08-26T21:54:02.837ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 4.020571ms iteration = 17 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
87232025-08-26T21:54:02.839ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
87242025-08-26T21:54:02.860ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 11 reason = Timeout
87252025-08-26T21:54:02.860ZDEBGnexus (BackgroundTasks): blueprint planning disabled, doing nothing background_task = blueprint_planner
87262025-08-26T21:54:02.860ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 12.32\u{b5}s iteration = 11 status = "Disabled"
87272025-08-26T21:54:02.860ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 9 reason = Timeout
87282025-08-26T21:54:02.860ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:53
87292025-08-26T21:54:02.860ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 9 reason = Timeout
87302025-08-26T21:54:02.860ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 9 reason = Timeout
87312025-08-26T21:54:02.860ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
87322025-08-26T21:54:02.860ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
87332025-08-26T21:54:02.860ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
87342025-08-26T21:54:02.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
87352025-08-26T21:54:02.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
87362025-08-26T21:54:02.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
87372025-08-26T21:54:02.861ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 9 reason = Timeout
87382025-08-26T21:54:02.861ZDEBGnexus (BackgroundTasks): activating background_task = alert_dispatcher iteration = 9 reason = Timeout
87392025-08-26T21:54:02.861ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
87402025-08-26T21:54:02.861ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
87412025-08-26T21:54:02.861ZDEBGnexus (BackgroundTasks): activating background_task = webhook_deliverator iteration = 9 reason = Timeout
87422025-08-26T21:54:02.861ZDEBGnexus (BackgroundTasks): roles background_task = webhook_deliverator roles = RoleSet { roles: {} }
87432025-08-26T21:54:02.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
87442025-08-26T21:54:02.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = alert_dispatcher resource = Database result = Ok(())
87452025-08-26T21:54:02.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = webhook_deliverator resource = Database result = Ok(())
87462025-08-26T21:54:02.863ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 2.284261ms iteration = 9 status = {"errors":[],"found_port_ok":[]}
87472025-08-26T21:54:02.863ZINFOnexus (BackgroundTasks): all webhook delivery tasks completed background_task = webhook_deliverator file = nexus/src/app/background/tasks/webhook_deliverator.rs:204 num_receivers = 0
87482025-08-26T21:54:02.863ZDEBGnexus (BackgroundTasks): activation complete background_task = webhook_deliverator elapsed = 2.376064ms iteration = 9 status = {"by_rx":{},"error":null}
87492025-08-26T21:54:02.865ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
87502025-08-26T21:54:02.866ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = alert_dispatcher resource = Database result = Ok(())
87512025-08-26T21:54:02.868ZDEBGnexus (BackgroundTasks): activation complete background_task = alert_dispatcher elapsed = 7.53697ms iteration = 9 status = {"dispatched":[],"errors":[],"glob_version":"183.0.0","globs_reprocessed":{},"no_receivers":[]}
87522025-08-26T21:54:02.869ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 17 reason = Timeout
87532025-08-26T21:54:02.869ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 17 reason = Timeout
87542025-08-26T21:54:02.869ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
87552025-08-26T21:54:02.869ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 17 reason = Timeout
87562025-08-26T21:54:02.869ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 20.44\u{b5}s iteration = 17 status = {"disabled":true,"errors":[],"sps":[]}
87572025-08-26T21:54:02.869ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
87582025-08-26T21:54:02.869ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 17 reason = Timeout
87592025-08-26T21:54:02.869ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 17 reason = Timeout
87602025-08-26T21:54:02.869ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
87612025-08-26T21:54:02.869ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
87622025-08-26T21:54:02.869ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 17 reason = Timeout
87632025-08-26T21:54:02.869ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 17 reason = Timeout
87642025-08-26T21:54:02.869ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
87652025-08-26T21:54:02.869ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
87662025-08-26T21:54:02.869ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
87672025-08-26T21:54:02.869ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 17 reason = Timeout
87682025-08-26T21:54:02.869ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
87692025-08-26T21:54:02.869ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 17 reason = Timeout
87702025-08-26T21:54:02.869ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
87712025-08-26T21:54:02.869ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 17 reason = Timeout
87722025-08-26T21:54:02.869ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
87732025-08-26T21:54:02.869ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 17 reason = Timeout
87742025-08-26T21:54:02.869ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
87752025-08-26T21:54:02.869ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
87762025-08-26T21:54:02.869ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 17 reason = Timeout
87772025-08-26T21:54:02.869ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
87782025-08-26T21:54:02.869ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 18 reason = Timeout
87792025-08-26T21:54:02.869ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
87802025-08-26T21:54:02.869ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 49 reason = Timeout
87812025-08-26T21:54:02.869ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
87822025-08-26T21:54:02.869ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 97 reason = Timeout
87832025-08-26T21:54:02.869ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
87842025-08-26T21:54:02.869ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 17 reason = Timeout
87852025-08-26T21:54:02.869ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
87862025-08-26T21:54:02.869ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 17 reason = Timeout
87872025-08-26T21:54:02.869ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
87882025-08-26T21:54:02.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
87892025-08-26T21:54:02.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
87902025-08-26T21:54:02.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
87912025-08-26T21:54:02.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
87922025-08-26T21:54:02.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
87932025-08-26T21:54:02.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
87942025-08-26T21:54:02.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
87952025-08-26T21:54:02.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
87962025-08-26T21:54:02.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
87972025-08-26T21:54:02.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
87982025-08-26T21:54:02.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
87992025-08-26T21:54:02.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
88002025-08-26T21:54:02.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
88012025-08-26T21:54:02.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
88022025-08-26T21:54:02.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
88032025-08-26T21:54:02.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
88042025-08-26T21:54:02.872ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 49 reason = Timeout
88052025-08-26T21:54:02.872ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
88062025-08-26T21:54:02.873ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
88072025-08-26T21:54:02.874ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
88082025-08-26T21:54:02.876ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
88092025-08-26T21:54:02.878ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
88102025-08-26T21:54:02.879ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
88112025-08-26T21:54:02.879ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
88122025-08-26T21:54:02.880ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
88132025-08-26T21:54:02.885ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
88142025-08-26T21:54:02.885ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
88152025-08-26T21:54:02.885ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
88162025-08-26T21:54:02.885ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
88172025-08-26T21:54:02.885ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
88182025-08-26T21:54:02.885ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
88192025-08-26T21:54:02.888ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
88202025-08-26T21:54:02.888ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
88212025-08-26T21:54:02.888ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
88222025-08-26T21:54:02.894ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
88232025-08-26T21:54:02.898ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
88242025-08-26T21:54:02.898ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
88252025-08-26T21:54:02.898ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
88262025-08-26T21:54:02.900ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
88272025-08-26T21:54:02.900ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
88282025-08-26T21:54:02.919ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
88292025-08-26T21:54:02.919ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:170 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
88302025-08-26T21:54:02.919ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 59.055121ms iteration = 9 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
88312025-08-26T21:54:02.924ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
88322025-08-26T21:54:02.926ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
88332025-08-26T21:54:02.926ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
88342025-08-26T21:54:02.928ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
88352025-08-26T21:54:02.929ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
88362025-08-26T21:54:02.929ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
88372025-08-26T21:54:02.929ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
88382025-08-26T21:54:02.929ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
88392025-08-26T21:54:02.929ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
88402025-08-26T21:54:02.930ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
88412025-08-26T21:54:02.930ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
88422025-08-26T21:54:02.930ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 61.68209ms iteration = 17 status = {"errors":[],"garbage_collect_requested":[]}
88432025-08-26T21:54:02.931ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
88442025-08-26T21:54:02.931ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
88452025-08-26T21:54:02.932ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
88462025-08-26T21:54:02.932ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
88472025-08-26T21:54:02.932ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
88482025-08-26T21:54:02.932ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
88492025-08-26T21:54:02.932ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
88502025-08-26T21:54:02.933ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
88512025-08-26T21:54:02.933ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
88522025-08-26T21:54:02.933ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
88532025-08-26T21:54:02.933ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
88542025-08-26T21:54:02.934ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
88552025-08-26T21:54:02.934ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
88562025-08-26T21:54:02.935ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
88572025-08-26T21:54:02.935ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
88582025-08-26T21:54:02.935ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 66.419667ms iteration = 17 status = {"errors":[],"finish_invoked_ok":[]}
88592025-08-26T21:54:02.935ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
88602025-08-26T21:54:02.936ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
88612025-08-26T21:54:02.936ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
88622025-08-26T21:54:02.936ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
88632025-08-26T21:54:02.937ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
88642025-08-26T21:54:02.940ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
88652025-08-26T21:54:02.940ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
88662025-08-26T21:54:02.940ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
88672025-08-26T21:54:02.940ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
88682025-08-26T21:54:02.940ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
88692025-08-26T21:54:02.940ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
88702025-08-26T21:54:02.940ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
88712025-08-26T21:54:02.940ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
88722025-08-26T21:54:02.941ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
88732025-08-26T21:54:02.941ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
88742025-08-26T21:54:02.941ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
88752025-08-26T21:54:02.941ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
88762025-08-26T21:54:02.941ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
88772025-08-26T21:54:02.941ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
88782025-08-26T21:54:02.942ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
88792025-08-26T21:54:02.942ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
88802025-08-26T21:54:02.943ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
88812025-08-26T21:54:02.943ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
88822025-08-26T21:54:02.944ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
88832025-08-26T21:54:02.944ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1346, tv_nsec: 802598037 } })
88842025-08-26T21:54:02.944ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1205 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
88852025-08-26T21:54:02.944ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 75.383459ms iteration = 49 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
88862025-08-26T21:54:02.946ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
88872025-08-26T21:54:02.946ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
88882025-08-26T21:54:02.946ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
88892025-08-26T21:54:02.948ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
88902025-08-26T21:54:02.948ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
88912025-08-26T21:54:02.948ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 78.929089ms iteration = 97 status = {"chicken_switches_updated":false}
88922025-08-26T21:54:02.949ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
88932025-08-26T21:54:02.949ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
88942025-08-26T21:54:02.950ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1213 zone_address = fd00:1122:3344:101::2
88952025-08-26T21:54:02.950ZDEBGnexus (MgsClient): client request background_task = nat_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
88962025-08-26T21:54:02.950ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 81.352825ms iteration = 17 status = {"errors":[],"requests_created_ok":[]}
88972025-08-26T21:54:02.950ZDEBGnexus (MgsClient): client response background_task = nat_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "686dfd35-5da4-4414-8cc1-486c2e609d79", "content-length": "26", "date": "Tue, 26 Aug 2025 21:54:02 GMT"} })
88982025-08-26T21:54:02.950ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1216 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
88992025-08-26T21:54:02.951ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
89002025-08-26T21:54:02.951ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
89012025-08-26T21:54:02.952ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
89022025-08-26T21:54:02.952ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
89032025-08-26T21:54:02.954ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
89042025-08-26T21:54:02.954ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
89052025-08-26T21:54:02.955ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1213 zone_address = fd00:1122:3344:101::2
89062025-08-26T21:54:02.955ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
89072025-08-26T21:54:02.955ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
89082025-08-26T21:54:02.955ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
89092025-08-26T21:54:02.955ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "9b12bfb0-7969-48d2-9991-64758343cf91", "content-length": "26", "date": "Tue, 26 Aug 2025 21:54:02 GMT"} })
89102025-08-26T21:54:02.956ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1216 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
89112025-08-26T21:54:02.957ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
89122025-08-26T21:54:02.959ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
89132025-08-26T21:54:02.959ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
89142025-08-26T21:54:02.959ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
89152025-08-26T21:54:02.967ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
89162025-08-26T21:54:02.967ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
89172025-08-26T21:54:02.967ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 94.734736ms iteration = 49 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
89182025-08-26T21:54:02.977ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
89192025-08-26T21:54:02.977ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
89202025-08-26T21:54:02.979ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
89212025-08-26T21:54:02.979ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 8e720f02-dde8-4423-a6e3-91698373805a dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
89222025-08-26T21:54:02.979ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
89232025-08-26T21:54:02.980ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
89242025-08-26T21:54:02.980ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
89252025-08-26T21:54:02.980ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
89262025-08-26T21:54:02.981ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
89272025-08-26T21:54:02.981ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
89282025-08-26T21:54:02.981ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
89292025-08-26T21:54:02.981ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
89302025-08-26T21:54:02.981ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
89312025-08-26T21:54:02.984ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
89322025-08-26T21:54:02.984ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
89332025-08-26T21:54:02.984ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
89342025-08-26T21:54:02.985ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 116.276491ms iteration = 17 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
89352025-08-26T21:54:02.985ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 116.618219ms iteration = 17 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
89362025-08-26T21:54:02.986ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
89372025-08-26T21:54:02.986ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
89382025-08-26T21:54:02.987ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 8e720f02-dde8-4423-a6e3-91698373805a dns_zone_name = control-plane.oxide.internal found_so_far = 59 version = 1
89392025-08-26T21:54:02.987ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
89402025-08-26T21:54:02.987ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2025-08-26 21:46:04.065217 UTC dns_group = internal generation = 1
89412025-08-26T21:54:02.987ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 153.898659ms iteration = 9 status = {"generation":1}
89422025-08-26T21:54:02.988ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
89432025-08-26T21:54:02.988ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
89442025-08-26T21:54:02.988ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
89452025-08-26T21:54:02.988ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
89462025-08-26T21:54:02.988ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
89472025-08-26T21:54:02.988ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
89482025-08-26T21:54:02.989ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
89492025-08-26T21:54:02.989ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
89502025-08-26T21:54:02.990ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
89512025-08-26T21:54:02.991ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
89522025-08-26T21:54:02.994ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
89532025-08-26T21:54:02.994ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
89542025-08-26T21:54:02.994ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: dd0c655a-95c2-48fc-b4c3-9733ed4b0faa, lookup_type: ById(dd0c655a-95c2-48fc-b4c3-9733ed4b0faa) } result = Ok(())
89552025-08-26T21:54:02.994ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
89562025-08-26T21:54:02.994ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
89572025-08-26T21:54:02.995ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
89582025-08-26T21:54:02.995ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:550 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
89592025-08-26T21:54:02.995ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 125.669212ms iteration = 17 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
89602025-08-26T21:54:02.996ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
89612025-08-26T21:54:02.997ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 128.161049ms iteration = 17 status = 0
89622025-08-26T21:54:03.002ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
89632025-08-26T21:54:03.002ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:138 found = 0 not_ready_to_be_deleted = 0
89642025-08-26T21:54:03.002ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 141.310625ms iteration = 9 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
89652025-08-26T21:54:03.002ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
89662025-08-26T21:54:03.002ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
89672025-08-26T21:54:03.002ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
89682025-08-26T21:54:03.012ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 143.599537ms iteration = 17 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
89692025-08-26T21:54:03.033ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
89702025-08-26T21:54:03.034ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
89712025-08-26T21:54:03.035ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
89722025-08-26T21:54:03.035ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
89732025-08-26T21:54:03.036ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
89742025-08-26T21:54:03.039ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
89752025-08-26T21:54:03.039ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
89762025-08-26T21:54:03.039ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
89772025-08-26T21:54:03.042ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
89782025-08-26T21:54:03.042ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
89792025-08-26T21:54:03.042ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 173.121743ms iteration = 17 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
89802025-08-26T21:54:03.043ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:407
89812025-08-26T21:54:03.043ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 174.410211ms iteration = 17 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
89822025-08-26T21:54:03.045ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
89832025-08-26T21:54:03.045ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
89842025-08-26T21:54:03.047ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
89852025-08-26T21:54:03.047ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = dfb404ab-acb8-43de-afe8-b429af0c161d dns_zone_name = oxide.test found_so_far = 0 version = 2
89862025-08-26T21:54:03.047ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
89872025-08-26T21:54:03.048ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
89882025-08-26T21:54:03.082ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
89892025-08-26T21:54:03.086ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
89902025-08-26T21:54:03.086ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
89912025-08-26T21:54:03.086ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
89922025-08-26T21:54:03.086ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1213 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 zone_address = fd00:1122:3344:101::2
89932025-08-26T21:54:03.086ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
89942025-08-26T21:54:03.087ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "f3d8b1ad-6d8f-4371-a70f-50e75d3db9a4", "content-length": "26", "date": "Tue, 26 Aug 2025 21:54:02 GMT"} })
89952025-08-26T21:54:03.087ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1216 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
89962025-08-26T21:54:03.087ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = dfb404ab-acb8-43de-afe8-b429af0c161d dns_zone_name = oxide.test found_so_far = 4 version = 2
89972025-08-26T21:54:03.087ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
89982025-08-26T21:54:03.087ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2025-08-26 21:46:05.296205 UTC dns_group = external generation = 2
89992025-08-26T21:54:03.087ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 254.436715ms iteration = 9 status = {"generation":2}
90002025-08-26T21:54:03.088ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
90012025-08-26T21:54:03.094ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-08-26T21:45:59.866217Z, time_modified: 2025-08-26T21:45:59.866217Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 9396aa55-de6f-4d6f-96db-d28e1c76fa9c, db_silo: Silo { identity: SiloIdentity { id: 9396aa55-de6f-4d6f-96db-d28e1c76fa9c, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-08-26T21:46:05.245521Z, time_modified: 2025-08-26T21:46:05.245521Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "e8529d720e4b0de099753dcdb373abb870af04baacd7c7dd850f5254858081af" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 9396aa55-de6f-4d6f-96db-d28e1c76fa9c, db_silo: Silo { identity: SiloIdentity { id: 9396aa55-de6f-4d6f-96db-d28e1c76fa9c, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-08-26T21:46:05.245521Z, time_modified: 2025-08-26T21:46:05.245521Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "e8529d720e4b0de099753dcdb373abb870af04baacd7c7dd850f5254858081af" }] }) }
90022025-08-26T21:54:03.094ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 260.942972ms iteration = 9 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"9396aa55-de6f-4d6f-96db-d28e1c76fa9c","tls_certs":["e8529d720e4b0de099753dcdb373abb870af04baacd7c7dd850f5254858081af"]}},"default_endpoint":{"silo_id":"9396aa55-de6f-4d6f-96db-d28e1c76fa9c","tls_certs":["e8529d720e4b0de099753dcdb373abb870af04baacd7c7dd850f5254858081af"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
90032025-08-26T21:54:03.109ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 239.769845ms iteration = 18 status = {"physical_disks_added":0}
90042025-08-26T21:54:03.122ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
90052025-08-26T21:54:03.168ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
90062025-08-26T21:54:03.168ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "f2be533c-ff50-4997-a315-206b410cfa13", "date": "Tue, 26 Aug 2025 21:54:03 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
90072025-08-26T21:54:03.168ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
90082025-08-26T21:54:03.169ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "d9391820-04fb-44d6-80ba-28453656d6ec", "date": "Tue, 26 Aug 2025 21:54:03 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
90092025-08-26T21:54:03.169ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
90102025-08-26T21:54:03.171ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
90112025-08-26T21:54:03.171ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 338.129762ms iteration = 11 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
90122025-08-26T21:54:03.176ZINFOnexus (BackgroundTasks): instance reincarnation completed successfully background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:113 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0
90132025-08-26T21:54:03.176ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 315.84093ms iteration = 9 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
90142025-08-26T21:54:03.176ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
90152025-08-26T21:54:03.177ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
90162025-08-26T21:54:03.179ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
90172025-08-26T21:54:03.181ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
90182025-08-26T21:54:03.181ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
90192025-08-26T21:54:03.181ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
90202025-08-26T21:54:03.182ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
90212025-08-26T21:54:03.184ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
90222025-08-26T21:54:03.184ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
90232025-08-26T21:54:03.184ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
90242025-08-26T21:54:03.252ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
90252025-08-26T21:54:03.253ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
90262025-08-26T21:54:03.255ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 422.116395ms iteration = 17 status = null
90272025-08-26T21:54:03.265ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 432.205243ms iteration = 17 status = {}
90282025-08-26T21:54:03.265ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
90292025-08-26T21:54:03.265ZINFOnexus (BackgroundTasks): VPC route manager sled dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
90302025-08-26T21:54:03.265ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
90312025-08-26T21:54:03.266ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "94c78195-63dc-4d9e-accc-e143947ce419", "date": "Tue, 26 Aug 2025 21:54:03 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
90322025-08-26T21:54:03.266ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
90332025-08-26T21:54:03.266ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "429279c6-fd54-46dc-8c59-76fcad65d2bf", "content-length": "372", "date": "Tue, 26 Aug 2025 21:54:03 GMT"} })
90342025-08-26T21:54:03.266ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
90352025-08-26T21:54:03.266ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
90362025-08-26T21:54:03.268ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "3dde713f-fe53-440a-800d-9626d590dadf", "date": "Tue, 26 Aug 2025 21:54:03 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
90372025-08-26T21:54:03.268ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
90382025-08-26T21:54:03.269ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "1b9d8ac1-a0c1-460c-b677-ea3acf062b84", "date": "Tue, 26 Aug 2025 21:54:03 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
90392025-08-26T21:54:03.269ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
90402025-08-26T21:54:03.271ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
90412025-08-26T21:54:03.271ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 438.559056ms iteration = 11 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
90422025-08-26T21:54:03.286ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
90432025-08-26T21:54:03.286ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
90442025-08-26T21:54:03.286ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
90452025-08-26T21:54:03.286ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "eb8f01f5-0bca-4cb2-9790-7d5c28ffed50", "date": "Tue, 26 Aug 2025 21:54:03 GMT"} })
90462025-08-26T21:54:03.286ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
90472025-08-26T21:54:03.287ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "309abc93-03d1-4488-b11d-4bceb5f9d629", "content-length": "691", "date": "Tue, 26 Aug 2025 21:54:03 GMT"} })
90482025-08-26T21:54:03.287ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
90492025-08-26T21:54:03.287ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
90502025-08-26T21:54:03.287ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
90512025-08-26T21:54:03.287ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
90522025-08-26T21:54:03.287ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 418.560765ms iteration = 17 status = {}
90532025-08-26T21:54:03.288ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
90542025-08-26T21:54:03.289ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
90552025-08-26T21:54:03.292ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
90562025-08-26T21:54:03.292ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
90572025-08-26T21:54:03.295ZINFOnexus (BackgroundTasks): VPC route manager sled dd0c655a-95c2-48fc-b4c3-9733ed4b0faa push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
90582025-08-26T21:54:03.295ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
90592025-08-26T21:54:03.295ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "2ff658d2-6a15-4775-a6d5-c4b79074a029", "date": "Tue, 26 Aug 2025 21:54:03 GMT"} })
90602025-08-26T21:54:03.297ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 427.942236ms iteration = 17 status = {}
90612025-08-26T21:54:03.329ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
90622025-08-26T21:54:03.329ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
90632025-08-26T21:54:03.331ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: d24c5c55-3a21-4a91-aeb4-45362e822df6, rack_id: e37a46f2-f789-4ca4-9841-f8c6dd705964, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(88d84f37-a4c8-4abc-af1f-27987b0d39d9) } rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0
90642025-08-26T21:54:03.331ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
90652025-08-26T21:54:03.331ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
90662025-08-26T21:54:03.342ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1832 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
90672025-08-26T21:54:03.342ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1859 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 88d84f37-a4c8-4abc-af1f-27987b0d39d9
90682025-08-26T21:54:03.342ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
90692025-08-26T21:54:03.342ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
90702025-08-26T21:54:03.342ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
90712025-08-26T21:54:03.342ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
90722025-08-26T21:54:03.342ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
90732025-08-26T21:54:03.342ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2006 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
90742025-08-26T21:54:03.342ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:408 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
90752025-08-26T21:54:03.342ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2042 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
90762025-08-26T21:54:03.343ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
90772025-08-26T21:54:03.343ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
90782025-08-26T21:54:03.343ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
90792025-08-26T21:54:03.345ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0
90802025-08-26T21:54:03.345ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
90812025-08-26T21:54:03.447ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "3222ec23-ebef-4e32-a86d-322745a04880", "date": "Tue, 26 Aug 2025 21:54:03 GMT"} })
90822025-08-26T21:54:03.447ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch = Switch0
90832025-08-26T21:54:03.447ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
90842025-08-26T21:54:03.447ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "3766ff5e-04d0-464c-97e1-f893303409d1", "content-length": "682", "date": "Tue, 26 Aug 2025 21:54:03 GMT"} })
90852025-08-26T21:54:03.447ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
90862025-08-26T21:54:03.448ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
90872025-08-26T21:54:03.451ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
90882025-08-26T21:54:03.451ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
90892025-08-26T21:54:03.455ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
90902025-08-26T21:54:03.455ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
90912025-08-26T21:54:03.457ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
90922025-08-26T21:54:03.457ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
90932025-08-26T21:54:03.464ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 595.174706ms iteration = 17 status = {}
90942025-08-26T21:54:05.352ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 330 local_addr = [fd00:1122:3344:101::c]:42016 method = GET remote_addr = [fd00:1122:3344:101::d]:37550 req_id = 9edcd61b-655c-489d-ae35-837ef7da25f6 response_code = 200 uri = /5d0959e5-47a6-4bf3-a52b-57950b9d900d
90952025-08-26T21:54:07.869ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 98 reason = Timeout
90962025-08-26T21:54:07.869ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
90972025-08-26T21:54:07.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
90982025-08-26T21:54:07.871ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
90992025-08-26T21:54:07.874ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
91002025-08-26T21:54:07.874ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
91012025-08-26T21:54:07.874ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
91022025-08-26T21:54:07.876ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
91032025-08-26T21:54:07.876ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.864037ms iteration = 98 status = {"chicken_switches_updated":false}
91042025-08-26T21:54:12.869ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 50 reason = Timeout
91052025-08-26T21:54:12.869ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
91062025-08-26T21:54:12.869ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 99 reason = Timeout
91072025-08-26T21:54:12.869ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
91082025-08-26T21:54:12.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
91092025-08-26T21:54:12.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
91102025-08-26T21:54:12.870ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
91112025-08-26T21:54:12.872ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 50 reason = Timeout
91122025-08-26T21:54:12.872ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
91132025-08-26T21:54:12.872ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
91142025-08-26T21:54:12.873ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
91152025-08-26T21:54:12.873ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
91162025-08-26T21:54:12.873ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
91172025-08-26T21:54:12.874ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
91182025-08-26T21:54:12.875ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
91192025-08-26T21:54:12.875ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 6.542688ms iteration = 99 status = {"chicken_switches_updated":false}
91202025-08-26T21:54:12.877ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
91212025-08-26T21:54:12.877ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
91222025-08-26T21:54:12.877ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
91232025-08-26T21:54:12.879ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
91242025-08-26T21:54:12.879ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
91252025-08-26T21:54:12.881ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
91262025-08-26T21:54:12.883ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
91272025-08-26T21:54:12.883ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
91282025-08-26T21:54:12.884ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
91292025-08-26T21:54:12.901ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
91302025-08-26T21:54:12.901ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 29.078936ms iteration = 50 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
91312025-08-26T21:54:12.909ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
91322025-08-26T21:54:12.909ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
91332025-08-26T21:54:12.944ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 75.89419ms iteration = 50 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
91342025-08-26T21:54:15.352ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 271 local_addr = [fd00:1122:3344:101::c]:42016 method = GET remote_addr = [fd00:1122:3344:101::d]:37550 req_id = aad95a03-23cc-46b9-b6f5-7ff58ed69a15 response_code = 200 uri = /5d0959e5-47a6-4bf3-a52b-57950b9d900d
91352025-08-26T21:54:17.869ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 100 reason = Timeout
91362025-08-26T21:54:17.870ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
91372025-08-26T21:54:17.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
91382025-08-26T21:54:17.872ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
91392025-08-26T21:54:17.875ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
91402025-08-26T21:54:17.875ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
91412025-08-26T21:54:17.875ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
91422025-08-26T21:54:17.877ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
91432025-08-26T21:54:17.877ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.734864ms iteration = 100 status = {"chicken_switches_updated":false}
91442025-08-26T21:54:22.869ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 51 reason = Timeout
91452025-08-26T21:54:22.869ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 101 reason = Timeout
91462025-08-26T21:54:22.869ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
91472025-08-26T21:54:22.869ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
91482025-08-26T21:54:22.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
91492025-08-26T21:54:22.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
91502025-08-26T21:54:22.871ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
91512025-08-26T21:54:22.872ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 51 reason = Timeout
91522025-08-26T21:54:22.872ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
91532025-08-26T21:54:22.872ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
91542025-08-26T21:54:22.874ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
91552025-08-26T21:54:22.874ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
91562025-08-26T21:54:22.874ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
91572025-08-26T21:54:22.874ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
91582025-08-26T21:54:22.875ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
91592025-08-26T21:54:22.875ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 6.883895ms iteration = 101 status = {"chicken_switches_updated":false}
91602025-08-26T21:54:22.877ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
91612025-08-26T21:54:22.877ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
91622025-08-26T21:54:22.877ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
91632025-08-26T21:54:22.879ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
91642025-08-26T21:54:22.879ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
91652025-08-26T21:54:22.880ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
91662025-08-26T21:54:22.883ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
91672025-08-26T21:54:22.883ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
91682025-08-26T21:54:22.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
91692025-08-26T21:54:22.906ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
91702025-08-26T21:54:22.906ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 33.642808ms iteration = 51 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
91712025-08-26T21:54:22.910ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
91722025-08-26T21:54:22.910ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
91732025-08-26T21:54:22.911ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 42.918408ms iteration = 51 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
91742025-08-26T21:54:25.353ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 453 local_addr = [fd00:1122:3344:101::c]:42016 method = GET remote_addr = [fd00:1122:3344:101::d]:37550 req_id = 7b9dd581-db51-4331-abd1-91d41cb0334e response_code = 200 uri = /5d0959e5-47a6-4bf3-a52b-57950b9d900d
91752025-08-26T21:54:27.869ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 102 reason = Timeout
91762025-08-26T21:54:27.869ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
91772025-08-26T21:54:27.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
91782025-08-26T21:54:27.871ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
91792025-08-26T21:54:27.874ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
91802025-08-26T21:54:27.874ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
91812025-08-26T21:54:27.874ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
91822025-08-26T21:54:27.876ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
91832025-08-26T21:54:27.876ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 6.730642ms iteration = 102 status = {"chicken_switches_updated":false}
91842025-08-26T21:54:32.833ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 18 reason = Timeout
91852025-08-26T21:54:32.833ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 18 reason = Timeout
91862025-08-26T21:54:32.833ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
91872025-08-26T21:54:32.833ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 18 reason = Timeout
91882025-08-26T21:54:32.833ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
91892025-08-26T21:54:32.833ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
91902025-08-26T21:54:32.833ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
91912025-08-26T21:54:32.836ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1376, tv_nsec: 694529391 } })
91922025-08-26T21:54:32.836ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1376, tv_nsec: 694651744 } })
91932025-08-26T21:54:32.836ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1205
91942025-08-26T21:54:32.836ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.662663ms iteration = 18 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
91952025-08-26T21:54:32.836ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1205
91962025-08-26T21:54:32.868ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 18 reason = Timeout
91972025-08-26T21:54:32.869ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 18 reason = Timeout
91982025-08-26T21:54:32.869ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
91992025-08-26T21:54:32.869ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
92002025-08-26T21:54:32.869ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
92012025-08-26T21:54:32.869ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 18 reason = Timeout
92022025-08-26T21:54:32.869ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 18 reason = Timeout
92032025-08-26T21:54:32.869ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
92042025-08-26T21:54:32.869ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
92052025-08-26T21:54:32.869ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 18 reason = Timeout
92062025-08-26T21:54:32.869ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
92072025-08-26T21:54:32.869ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 18 reason = Timeout
92082025-08-26T21:54:32.869ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
92092025-08-26T21:54:32.869ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 52 reason = Timeout
92102025-08-26T21:54:32.869ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
92112025-08-26T21:54:32.869ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
92122025-08-26T21:54:32.869ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 103 reason = Timeout
92132025-08-26T21:54:32.869ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
92142025-08-26T21:54:32.869ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 19 reason = Timeout
92152025-08-26T21:54:32.869ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 18 reason = Timeout
92162025-08-26T21:54:32.869ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
92172025-08-26T21:54:32.869ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
92182025-08-26T21:54:32.869ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 18 reason = Timeout
92192025-08-26T21:54:32.869ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
92202025-08-26T21:54:32.869ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 18 reason = Timeout
92212025-08-26T21:54:32.869ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
92222025-08-26T21:54:32.869ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 18 reason = Timeout
92232025-08-26T21:54:32.869ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 12.19\u{b5}s iteration = 18 status = {"disabled":true,"errors":[],"sps":[]}
92242025-08-26T21:54:32.869ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 18 reason = Timeout
92252025-08-26T21:54:32.869ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 18 reason = Timeout
92262025-08-26T21:54:32.869ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
92272025-08-26T21:54:32.869ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 18 reason = Timeout
92282025-08-26T21:54:32.869ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
92292025-08-26T21:54:32.869ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
92302025-08-26T21:54:32.869ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 18 reason = Timeout
92312025-08-26T21:54:32.869ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
92322025-08-26T21:54:32.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
92332025-08-26T21:54:32.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
92342025-08-26T21:54:32.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
92352025-08-26T21:54:32.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
92362025-08-26T21:54:32.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
92372025-08-26T21:54:32.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
92382025-08-26T21:54:32.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
92392025-08-26T21:54:32.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
92402025-08-26T21:54:32.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
92412025-08-26T21:54:32.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
92422025-08-26T21:54:32.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
92432025-08-26T21:54:32.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
92442025-08-26T21:54:32.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
92452025-08-26T21:54:32.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
92462025-08-26T21:54:32.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
92472025-08-26T21:54:32.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
92482025-08-26T21:54:32.871ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
92492025-08-26T21:54:32.871ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
92502025-08-26T21:54:32.871ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
92512025-08-26T21:54:32.871ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
92522025-08-26T21:54:32.871ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
92532025-08-26T21:54:32.871ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
92542025-08-26T21:54:32.871ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
92552025-08-26T21:54:32.872ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 52 reason = Timeout
92562025-08-26T21:54:32.872ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
92572025-08-26T21:54:32.873ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
92582025-08-26T21:54:32.873ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
92592025-08-26T21:54:32.873ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
92602025-08-26T21:54:32.874ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
92612025-08-26T21:54:32.875ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 6.561458ms iteration = 18 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
92622025-08-26T21:54:32.875ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
92632025-08-26T21:54:32.875ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
92642025-08-26T21:54:32.876ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
92652025-08-26T21:54:32.876ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
92662025-08-26T21:54:32.876ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
92672025-08-26T21:54:32.876ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
92682025-08-26T21:54:32.876ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
92692025-08-26T21:54:32.876ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
92702025-08-26T21:54:32.876ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
92712025-08-26T21:54:32.876ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
92722025-08-26T21:54:32.876ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
92732025-08-26T21:54:32.876ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
92742025-08-26T21:54:32.876ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
92752025-08-26T21:54:32.877ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
92762025-08-26T21:54:32.877ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
92772025-08-26T21:54:32.879ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
92782025-08-26T21:54:32.879ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
92792025-08-26T21:54:32.879ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
92802025-08-26T21:54:32.882ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
92812025-08-26T21:54:32.882ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
92822025-08-26T21:54:32.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
92832025-08-26T21:54:32.882ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
92842025-08-26T21:54:32.883ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
92852025-08-26T21:54:32.883ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
92862025-08-26T21:54:32.884ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
92872025-08-26T21:54:32.884ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
92882025-08-26T21:54:32.884ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1376, tv_nsec: 743279980 } })
92892025-08-26T21:54:32.884ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
92902025-08-26T21:54:32.884ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
92912025-08-26T21:54:32.885ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
92922025-08-26T21:54:32.885ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1205 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
92932025-08-26T21:54:32.886ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
92942025-08-26T21:54:32.886ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
92952025-08-26T21:54:32.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
92962025-08-26T21:54:32.887ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
92972025-08-26T21:54:32.887ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 18.285512ms iteration = 103 status = {"chicken_switches_updated":false}
92982025-08-26T21:54:32.887ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
92992025-08-26T21:54:32.888ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
93002025-08-26T21:54:32.889ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
93012025-08-26T21:54:32.890ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
93022025-08-26T21:54:32.893ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
93032025-08-26T21:54:32.893ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
93042025-08-26T21:54:32.893ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
93052025-08-26T21:54:32.894ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
93062025-08-26T21:54:32.894ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
93072025-08-26T21:54:32.894ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
93082025-08-26T21:54:32.916ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
93092025-08-26T21:54:32.916ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 44.166936ms iteration = 52 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
93102025-08-26T21:54:32.920ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
93112025-08-26T21:54:32.921ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
93122025-08-26T21:54:32.921ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 52.310119ms iteration = 18 status = {"errors":[],"requests_created_ok":[]}
93132025-08-26T21:54:32.922ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 53.179119ms iteration = 18 status = {"errors":[],"garbage_collect_requested":[]}
93142025-08-26T21:54:32.923ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 54.839766ms iteration = 18 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
93152025-08-26T21:54:32.926ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
93162025-08-26T21:54:32.926ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
93172025-08-26T21:54:32.926ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
93182025-08-26T21:54:32.926ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
93192025-08-26T21:54:32.928ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 59.069652ms iteration = 18 status = {"errors":[],"finish_invoked_ok":[]}
93202025-08-26T21:54:32.928ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
93212025-08-26T21:54:32.928ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
93222025-08-26T21:54:32.934ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
93232025-08-26T21:54:32.935ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
93242025-08-26T21:54:32.935ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
93252025-08-26T21:54:32.935ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
93262025-08-26T21:54:32.938ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
93272025-08-26T21:54:32.939ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
93282025-08-26T21:54:32.941ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
93292025-08-26T21:54:32.943ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:550 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
93302025-08-26T21:54:32.943ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 74.754155ms iteration = 18 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
93312025-08-26T21:54:32.945ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: dd0c655a-95c2-48fc-b4c3-9733ed4b0faa, lookup_type: ById(dd0c655a-95c2-48fc-b4c3-9733ed4b0faa) } result = Ok(())
93322025-08-26T21:54:32.945ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
93332025-08-26T21:54:32.945ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
93342025-08-26T21:54:32.948ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
93352025-08-26T21:54:32.948ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 79.169994ms iteration = 18 status = 0
93362025-08-26T21:54:32.948ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
93372025-08-26T21:54:32.949ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
93382025-08-26T21:54:32.952ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
93392025-08-26T21:54:32.952ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
93402025-08-26T21:54:32.952ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
93412025-08-26T21:54:32.973ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
93422025-08-26T21:54:32.973ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
93432025-08-26T21:54:32.973ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
93442025-08-26T21:54:32.973ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
93452025-08-26T21:54:32.984ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
93462025-08-26T21:54:32.984ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
93472025-08-26T21:54:32.986ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
93482025-08-26T21:54:32.986ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
93492025-08-26T21:54:32.986ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 117.647242ms iteration = 52 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
93502025-08-26T21:54:32.987ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1213 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 zone_address = fd00:1122:3344:101::2
93512025-08-26T21:54:32.987ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
93522025-08-26T21:54:32.988ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "95e5397d-2474-40ab-9427-48134e9b4443", "content-length": "26", "date": "Tue, 26 Aug 2025 21:54:32 GMT"} })
93532025-08-26T21:54:32.988ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1216 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
93542025-08-26T21:54:32.990ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
93552025-08-26T21:54:33.002ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:407
93562025-08-26T21:54:33.002ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 132.874985ms iteration = 18 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
93572025-08-26T21:54:33.015ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
93582025-08-26T21:54:33.016ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
93592025-08-26T21:54:33.016ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 147.031525ms iteration = 18 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
93602025-08-26T21:54:33.018ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 149.699884ms iteration = 19 status = {"physical_disks_added":0}
93612025-08-26T21:54:33.019ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 150.068913ms iteration = 18 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
93622025-08-26T21:54:33.036ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
93632025-08-26T21:54:33.037ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
93642025-08-26T21:54:33.038ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
93652025-08-26T21:54:33.040ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
93662025-08-26T21:54:33.040ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
93672025-08-26T21:54:33.040ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
93682025-08-26T21:54:33.042ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
93692025-08-26T21:54:33.042ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1213 zone_address = fd00:1122:3344:101::2
93702025-08-26T21:54:33.042ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
93712025-08-26T21:54:33.042ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "9745ad88-daa1-4903-bae3-927f61fac007", "content-length": "26", "date": "Tue, 26 Aug 2025 21:54:32 GMT"} })
93722025-08-26T21:54:33.042ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1216 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
93732025-08-26T21:54:33.044ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
93742025-08-26T21:54:33.044ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
93752025-08-26T21:54:33.044ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
93762025-08-26T21:54:33.044ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1213 zone_address = fd00:1122:3344:101::2
93772025-08-26T21:54:33.044ZDEBGnexus (MgsClient): client request background_task = nat_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
93782025-08-26T21:54:33.044ZDEBGnexus (MgsClient): client response background_task = nat_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "5334f3ca-3726-4f79-991c-5b6c38a1c665", "content-length": "26", "date": "Tue, 26 Aug 2025 21:54:32 GMT"} })
93792025-08-26T21:54:33.044ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1216 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
93802025-08-26T21:54:33.045ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
93812025-08-26T21:54:33.047ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
93822025-08-26T21:54:33.096ZINFOnexus (BackgroundTasks): VPC route manager sled dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
93832025-08-26T21:54:33.096ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
93842025-08-26T21:54:33.096ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "8cd56c59-99e0-4e3e-9ea2-f56fe41036e8", "content-length": "372", "date": "Tue, 26 Aug 2025 21:54:33 GMT"} })
93852025-08-26T21:54:33.096ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
93862025-08-26T21:54:33.096ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
93872025-08-26T21:54:33.150ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
93882025-08-26T21:54:33.150ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
93892025-08-26T21:54:33.151ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "597a188a-96c0-460e-9d4a-f731399dde56", "date": "Tue, 26 Aug 2025 21:54:33 GMT"} })
93902025-08-26T21:54:33.151ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
93912025-08-26T21:54:33.151ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
93922025-08-26T21:54:33.153ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
93932025-08-26T21:54:33.154ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
93942025-08-26T21:54:33.157ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
93952025-08-26T21:54:33.157ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
93962025-08-26T21:54:33.160ZINFOnexus (BackgroundTasks): VPC route manager sled dd0c655a-95c2-48fc-b4c3-9733ed4b0faa push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
93972025-08-26T21:54:33.160ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
93982025-08-26T21:54:33.161ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "c8643f19-433e-4115-8961-a469d5e77868", "date": "Tue, 26 Aug 2025 21:54:33 GMT"} })
93992025-08-26T21:54:33.163ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 294.195732ms iteration = 18 status = {}
94002025-08-26T21:54:33.182ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 348.727251ms iteration = 18 status = {}
94012025-08-26T21:54:33.186ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
94022025-08-26T21:54:33.186ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "0683e6c0-4737-4ea1-8f1c-db0dcffd0b48", "content-length": "691", "date": "Tue, 26 Aug 2025 21:54:33 GMT"} })
94032025-08-26T21:54:33.186ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
94042025-08-26T21:54:33.186ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
94052025-08-26T21:54:33.186ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
94062025-08-26T21:54:33.188ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 319.337809ms iteration = 18 status = {}
94072025-08-26T21:54:33.252ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
94082025-08-26T21:54:33.253ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
94092025-08-26T21:54:33.255ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 421.751017ms iteration = 18 status = null
94102025-08-26T21:54:33.295ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
94112025-08-26T21:54:33.295ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
94122025-08-26T21:54:33.297ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: d24c5c55-3a21-4a91-aeb4-45362e822df6, rack_id: e37a46f2-f789-4ca4-9841-f8c6dd705964, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(88d84f37-a4c8-4abc-af1f-27987b0d39d9) } rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0
94132025-08-26T21:54:33.297ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
94142025-08-26T21:54:33.298ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
94152025-08-26T21:54:33.309ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1832 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
94162025-08-26T21:54:33.309ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1859 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 88d84f37-a4c8-4abc-af1f-27987b0d39d9
94172025-08-26T21:54:33.309ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
94182025-08-26T21:54:33.309ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
94192025-08-26T21:54:33.309ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
94202025-08-26T21:54:33.309ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
94212025-08-26T21:54:33.309ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
94222025-08-26T21:54:33.309ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2006 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
94232025-08-26T21:54:33.309ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:408 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
94242025-08-26T21:54:33.309ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2042 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
94252025-08-26T21:54:33.310ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
94262025-08-26T21:54:33.310ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
94272025-08-26T21:54:33.310ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
94282025-08-26T21:54:33.312ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0
94292025-08-26T21:54:33.312ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
94302025-08-26T21:54:33.419ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "b9dcb256-3bc5-4013-bc05-6b3f96488162", "date": "Tue, 26 Aug 2025 21:54:33 GMT"} })
94312025-08-26T21:54:33.419ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch = Switch0
94322025-08-26T21:54:33.419ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
94332025-08-26T21:54:33.419ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "d0ca47d2-9b50-4151-b06f-e8e056862086", "content-length": "682", "date": "Tue, 26 Aug 2025 21:54:33 GMT"} })
94342025-08-26T21:54:33.419ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
94352025-08-26T21:54:33.419ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
94362025-08-26T21:54:33.423ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
94372025-08-26T21:54:33.423ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
94382025-08-26T21:54:33.426ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
94392025-08-26T21:54:33.427ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
94402025-08-26T21:54:33.428ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
94412025-08-26T21:54:33.429ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
94422025-08-26T21:54:33.435ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 566.472659ms iteration = 18 status = {}
94432025-08-26T21:54:33.495ZDEBGnexus (producer-registration-task): registering as oximeter producer with Nexus registration_count = 3
94442025-08-26T21:54:33.495ZDEBGnexus (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::c]:12221
94452025-08-26T21:54:33.526ZDEBGnexus (producer-registration-task): client request body = Some(Body) method = POST uri = http://[fd00:1122:3344:101::c]:12221/metrics/producers
94462025-08-26T21:54:33.526ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:1025 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::c]:54782
94472025-08-26T21:54:33.527ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:54782 req_id = f52438fc-a079-483e-b2f6-4aec812968da roles = RoleSet { roles: {} } uri = /metrics/producers
94482025-08-26T21:54:33.527ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:54782 req_id = f52438fc-a079-483e-b2f6-4aec812968da resource = Database result = Ok(()) uri = /metrics/producers
94492025-08-26T21:54:33.575ZINFOnexus: assigned collector to new producer collector_id = f843ff0e-eef8-49c4-b428-23e15df1def3 file = nexus/src/app/oximeter.rs:76 producer_id = 5d0959e5-47a6-4bf3-a52b-57950b9d900d
94502025-08-26T21:54:33.575ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 48769 local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:54782 req_id = f52438fc-a079-483e-b2f6-4aec812968da response_code = 201 uri = /metrics/producers
94512025-08-26T21:54:33.575ZDEBGnexus (producer-registration-task): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::c]:12221/metrics/producers", status: 201, headers: {"content-type": "application/json", "x-request-id": "f52438fc-a079-483e-b2f6-4aec812968da", "content-length": "41", "date": "Tue, 26 Aug 2025 21:54:33 GMT"} })
94522025-08-26T21:54:33.577ZDEBGnexus (producer-registration-task): successfully registered with Nexus, pausing until time to renew lease lease_duration = 600s wait_period = 150s
94532025-08-26T21:54:35.352ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 314 local_addr = [fd00:1122:3344:101::c]:42016 method = GET remote_addr = [fd00:1122:3344:101::d]:37550 req_id = 9d9fa7ca-dc4d-4bff-a9b2-32acb031bbdd response_code = 200 uri = /5d0959e5-47a6-4bf3-a52b-57950b9d900d
94542025-08-26T21:54:37.869ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 104 reason = Timeout
94552025-08-26T21:54:37.869ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
94562025-08-26T21:54:37.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
94572025-08-26T21:54:37.871ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
94582025-08-26T21:54:37.874ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
94592025-08-26T21:54:37.874ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
94602025-08-26T21:54:37.874ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
94612025-08-26T21:54:37.876ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
94622025-08-26T21:54:37.876ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.314445ms iteration = 104 status = {"chicken_switches_updated":false}
94632025-08-26T21:54:42.869ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 53 reason = Timeout
94642025-08-26T21:54:42.869ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
94652025-08-26T21:54:42.869ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 105 reason = Timeout
94662025-08-26T21:54:42.869ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
94672025-08-26T21:54:42.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
94682025-08-26T21:54:42.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
94692025-08-26T21:54:42.871ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
94702025-08-26T21:54:42.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
94712025-08-26T21:54:42.871ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
94722025-08-26T21:54:42.872ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 53 reason = Timeout
94732025-08-26T21:54:42.872ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
94742025-08-26T21:54:42.872ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
94752025-08-26T21:54:42.872ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.12825ms iteration = 53 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
94762025-08-26T21:54:42.873ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
94772025-08-26T21:54:42.874ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
94782025-08-26T21:54:42.874ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
94792025-08-26T21:54:42.874ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
94802025-08-26T21:54:42.876ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
94812025-08-26T21:54:42.876ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 6.64299ms iteration = 105 status = {"chicken_switches_updated":false}
94822025-08-26T21:54:42.876ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
94832025-08-26T21:54:42.876ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
94842025-08-26T21:54:42.876ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
94852025-08-26T21:54:42.878ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
94862025-08-26T21:54:42.878ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
94872025-08-26T21:54:42.879ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
94882025-08-26T21:54:42.882ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
94892025-08-26T21:54:42.882ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
94902025-08-26T21:54:42.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
94912025-08-26T21:54:42.898ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
94922025-08-26T21:54:42.898ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 26.019926ms iteration = 53 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
94932025-08-26T21:54:45.352ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 286 local_addr = [fd00:1122:3344:101::c]:42016 method = GET remote_addr = [fd00:1122:3344:101::d]:37550 req_id = 7aa372a8-d4d0-4e7c-bc65-cf539de3d950 response_code = 200 uri = /5d0959e5-47a6-4bf3-a52b-57950b9d900d
94942025-08-26T21:54:47.869ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 106 reason = Timeout
94952025-08-26T21:54:47.869ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
94962025-08-26T21:54:47.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
94972025-08-26T21:54:47.872ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
94982025-08-26T21:54:47.875ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
94992025-08-26T21:54:47.875ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
95002025-08-26T21:54:47.875ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
95012025-08-26T21:54:47.877ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
95022025-08-26T21:54:47.877ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 8.441931ms iteration = 106 status = {"chicken_switches_updated":false}
95032025-08-26T21:54:52.869ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 54 reason = Timeout
95042025-08-26T21:54:52.869ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
95052025-08-26T21:54:52.869ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 107 reason = Timeout
95062025-08-26T21:54:52.869ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
95072025-08-26T21:54:52.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
95082025-08-26T21:54:52.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
95092025-08-26T21:54:52.871ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 54 reason = Timeout
95102025-08-26T21:54:52.871ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
95112025-08-26T21:54:52.871ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
95122025-08-26T21:54:52.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
95132025-08-26T21:54:52.873ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
95142025-08-26T21:54:52.874ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
95152025-08-26T21:54:52.874ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
95162025-08-26T21:54:52.874ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
95172025-08-26T21:54:52.876ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
95182025-08-26T21:54:52.876ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
95192025-08-26T21:54:52.876ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
95202025-08-26T21:54:52.877ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
95212025-08-26T21:54:52.877ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.812807ms iteration = 107 status = {"chicken_switches_updated":false}
95222025-08-26T21:54:52.878ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
95232025-08-26T21:54:52.879ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
95242025-08-26T21:54:52.880ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
95252025-08-26T21:54:52.883ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
95262025-08-26T21:54:52.883ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
95272025-08-26T21:54:52.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
95282025-08-26T21:54:52.896ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
95292025-08-26T21:54:52.896ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 24.909072ms iteration = 54 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
95302025-08-26T21:54:52.908ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
95312025-08-26T21:54:52.908ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
95322025-08-26T21:54:52.910ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 40.862271ms iteration = 54 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
95332025-08-26T21:54:55.352ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 518 local_addr = [fd00:1122:3344:101::c]:42016 method = GET remote_addr = [fd00:1122:3344:101::d]:37550 req_id = a370dd17-7de5-46be-9157-f570bad3de7a response_code = 200 uri = /5d0959e5-47a6-4bf3-a52b-57950b9d900d
95342025-08-26T21:54:57.869ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 108 reason = Timeout
95352025-08-26T21:54:57.869ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
95362025-08-26T21:54:57.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
95372025-08-26T21:54:57.871ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
95382025-08-26T21:54:57.874ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
95392025-08-26T21:54:57.874ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
95402025-08-26T21:54:57.874ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
95412025-08-26T21:54:57.876ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
95422025-08-26T21:54:57.876ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.061159ms iteration = 108 status = {"chicken_switches_updated":false}
95432025-08-26T21:55:02.832ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 10 reason = Timeout
95442025-08-26T21:55:02.832ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 10 reason = Timeout
95452025-08-26T21:55:02.832ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 10 reason = Timeout
95462025-08-26T21:55:02.832ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
95472025-08-26T21:55:02.832ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 10 reason = Timeout
95482025-08-26T21:55:02.832ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-08-26 21:45:02.832819995 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
95492025-08-26T21:55:02.832ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
95502025-08-26T21:55:02.832ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
95512025-08-26T21:55:02.832ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
95522025-08-26T21:55:02.832ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 10 reason = Timeout
95532025-08-26T21:55:02.832ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
95542025-08-26T21:55:02.832ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
95552025-08-26T21:55:02.832ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 10 reason = Timeout
95562025-08-26T21:55:02.832ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 19 reason = Timeout
95572025-08-26T21:55:02.832ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
95582025-08-26T21:55:02.832ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 19 reason = Timeout
95592025-08-26T21:55:02.832ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
95602025-08-26T21:55:02.832ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 12 reason = Timeout
95612025-08-26T21:55:02.833ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 19 reason = Timeout
95622025-08-26T21:55:02.833ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 12 reason = Timeout
95632025-08-26T21:55:02.833ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
95642025-08-26T21:55:02.833ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
95652025-08-26T21:55:02.833ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
95662025-08-26T21:55:02.833ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
95672025-08-26T21:55:02.833ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
95682025-08-26T21:55:02.833ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
95692025-08-26T21:55:02.833ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("60abf731-31e1-4111-a3e2-65da89eaed87.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("84c849eb-0949-41d9-ad02-9a5509144564.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("84c849eb-0949-41d9-ad02-9a5509144564.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::8)) }], valid_until: Instant { tv_sec: 1406, tv_nsec: 692032075 } })
95702025-08-26T21:55:02.833ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("852fd1a3-de9a-47e8-a667-5e8dab880ad3.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("c68f1501-294c-4874-92ca-8fc0546a5870.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("d741a72f-8745-4723-9e80-e1e72987266b.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("d741a72f-8745-4723-9e80-e1e72987266b.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:3::1)) }], valid_until: Instant { tv_sec: 1406, tv_nsec: 692035295 } })
95712025-08-26T21:55:02.834ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
95722025-08-26T21:55:02.834ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.238308ms iteration = 10 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
95732025-08-26T21:55:02.834ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } dns_group = internal
95742025-08-26T21:55:02.834ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.166666ms iteration = 10 status = {"addresses":["[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:3::1]:5353"]}
95752025-08-26T21:55:02.834ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
95762025-08-26T21:55:02.834ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
95772025-08-26T21:55:02.834ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
95782025-08-26T21:55:02.834ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
95792025-08-26T21:55:02.835ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-08-26 21:45:02.832819995 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
95802025-08-26T21:55:02.835ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 2.830714ms iteration = 10 status = {"expiration":"2025-08-26T21:45:02.832819995Z","pruned":[]}
95812025-08-26T21:55:02.835ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 11 reason = Timeout
95822025-08-26T21:55:02.835ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:91 reason = blueprint disabled target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8
95832025-08-26T21:55:02.835ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 27.61\u{b5}s iteration = 11 status = {"enabled":false,"target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8"}
95842025-08-26T21:55:02.835ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1406, tv_nsec: 694444310 } })
95852025-08-26T21:55:02.836ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1205
95862025-08-26T21:55:02.836ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1406, tv_nsec: 694832558 } })
95872025-08-26T21:55:02.836ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1205
95882025-08-26T21:55:02.836ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.812395ms iteration = 19 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
95892025-08-26T21:55:02.860ZDEBGnexus (BackgroundTasks): activating background_task = crdb_node_id_collector iteration = 5 reason = Timeout
95902025-08-26T21:55:02.860ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 10 reason = Timeout
95912025-08-26T21:55:02.860ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 12 reason = Timeout
95922025-08-26T21:55:02.860ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
95932025-08-26T21:55:02.860ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
95942025-08-26T21:55:02.860ZDEBGnexus (BackgroundTasks): blueprint planning disabled, doing nothing background_task = blueprint_planner
95952025-08-26T21:55:02.860ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 11.951\u{b5}s iteration = 12 status = "Disabled"
95962025-08-26T21:55:02.860ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 10 reason = Timeout
95972025-08-26T21:55:02.860ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:53
95982025-08-26T21:55:02.860ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 10 reason = Timeout
95992025-08-26T21:55:02.860ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
96002025-08-26T21:55:02.860ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
96012025-08-26T21:55:02.860ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
96022025-08-26T21:55:02.860ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
96032025-08-26T21:55:02.860ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
96042025-08-26T21:55:02.860ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
96052025-08-26T21:55:02.860ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
96062025-08-26T21:55:02.860ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 10 reason = Timeout
96072025-08-26T21:55:02.861ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
96082025-08-26T21:55:02.861ZDEBGnexus (BackgroundTasks): activating background_task = alert_dispatcher iteration = 10 reason = Timeout
96092025-08-26T21:55:02.861ZDEBGnexus (BackgroundTasks): activating background_task = webhook_deliverator iteration = 10 reason = Timeout
96102025-08-26T21:55:02.861ZDEBGnexus (BackgroundTasks): roles background_task = webhook_deliverator roles = RoleSet { roles: {} }
96112025-08-26T21:55:02.861ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
96122025-08-26T21:55:02.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
96132025-08-26T21:55:02.861ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
96142025-08-26T21:55:02.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
96152025-08-26T21:55:02.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = alert_dispatcher resource = Database result = Ok(())
96162025-08-26T21:55:02.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = webhook_deliverator resource = Database result = Ok(())
96172025-08-26T21:55:02.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
96182025-08-26T21:55:02.861ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
96192025-08-26T21:55:02.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
96202025-08-26T21:55:02.861ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
96212025-08-26T21:55:02.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
96222025-08-26T21:55:02.863ZINFOnexus (BackgroundTasks): all webhook delivery tasks completed background_task = webhook_deliverator file = nexus/src/app/background/tasks/webhook_deliverator.rs:204 num_receivers = 0
96232025-08-26T21:55:02.863ZDEBGnexus (BackgroundTasks): activation complete background_task = webhook_deliverator elapsed = 2.601929ms iteration = 10 status = {"by_rx":{},"error":null}
96242025-08-26T21:55:02.863ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
96252025-08-26T21:55:02.864ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 3.210562ms iteration = 10 status = {"errors":[],"found_port_ok":[]}
96262025-08-26T21:55:02.865ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
96272025-08-26T21:55:02.866ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = alert_dispatcher resource = Database result = Ok(())
96282025-08-26T21:55:02.867ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
96292025-08-26T21:55:02.867ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
96302025-08-26T21:55:02.867ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
96312025-08-26T21:55:02.867ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
96322025-08-26T21:55:02.868ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 19 reason = Timeout
96332025-08-26T21:55:02.868ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 18.451\u{b5}s iteration = 19 status = {"disabled":true,"errors":[],"sps":[]}
96342025-08-26T21:55:02.868ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 19 reason = Timeout
96352025-08-26T21:55:02.868ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 19 reason = Timeout
96362025-08-26T21:55:02.868ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
96372025-08-26T21:55:02.868ZDEBGnexus (BackgroundTasks): activation complete background_task = alert_dispatcher elapsed = 7.788506ms iteration = 10 status = {"dispatched":[],"errors":[],"glob_version":"183.0.0","globs_reprocessed":{},"no_receivers":[]}
96382025-08-26T21:55:02.868ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 19 reason = Timeout
96392025-08-26T21:55:02.868ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
96402025-08-26T21:55:02.868ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
96412025-08-26T21:55:02.868ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
96422025-08-26T21:55:02.868ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 19 reason = Timeout
96432025-08-26T21:55:02.868ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 19 reason = Timeout
96442025-08-26T21:55:02.868ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
96452025-08-26T21:55:02.868ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 19 reason = Timeout
96462025-08-26T21:55:02.868ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
96472025-08-26T21:55:02.868ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
96482025-08-26T21:55:02.868ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 19 reason = Timeout
96492025-08-26T21:55:02.868ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 55 reason = Timeout
96502025-08-26T21:55:02.868ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
96512025-08-26T21:55:02.868ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 19 reason = Timeout
96522025-08-26T21:55:02.868ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
96532025-08-26T21:55:02.868ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 109 reason = Timeout
96542025-08-26T21:55:02.868ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
96552025-08-26T21:55:02.868ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
96562025-08-26T21:55:02.868ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 19 reason = Timeout
96572025-08-26T21:55:02.869ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
96582025-08-26T21:55:02.869ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 20 reason = Timeout
96592025-08-26T21:55:02.869ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
96602025-08-26T21:55:02.869ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 19 reason = Timeout
96612025-08-26T21:55:02.869ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
96622025-08-26T21:55:02.869ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
96632025-08-26T21:55:02.869ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 19 reason = Timeout
96642025-08-26T21:55:02.869ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
96652025-08-26T21:55:02.869ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 19 reason = Timeout
96662025-08-26T21:55:02.869ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
96672025-08-26T21:55:02.869ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 19 reason = Timeout
96682025-08-26T21:55:02.869ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
96692025-08-26T21:55:02.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
96702025-08-26T21:55:02.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
96712025-08-26T21:55:02.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
96722025-08-26T21:55:02.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
96732025-08-26T21:55:02.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
96742025-08-26T21:55:02.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
96752025-08-26T21:55:02.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
96762025-08-26T21:55:02.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
96772025-08-26T21:55:02.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
96782025-08-26T21:55:02.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
96792025-08-26T21:55:02.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
96802025-08-26T21:55:02.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
96812025-08-26T21:55:02.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
96822025-08-26T21:55:02.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
96832025-08-26T21:55:02.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
96842025-08-26T21:55:02.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
96852025-08-26T21:55:02.871ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
96862025-08-26T21:55:02.871ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
96872025-08-26T21:55:02.871ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
96882025-08-26T21:55:02.871ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
96892025-08-26T21:55:02.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
96902025-08-26T21:55:02.871ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
96912025-08-26T21:55:02.872ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 55 reason = Timeout
96922025-08-26T21:55:02.872ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
96932025-08-26T21:55:02.872ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
96942025-08-26T21:55:02.872ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
96952025-08-26T21:55:02.873ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
96962025-08-26T21:55:02.873ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
96972025-08-26T21:55:02.874ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
96982025-08-26T21:55:02.875ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
96992025-08-26T21:55:02.876ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
97002025-08-26T21:55:02.876ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
97012025-08-26T21:55:02.876ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
97022025-08-26T21:55:02.876ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
97032025-08-26T21:55:02.876ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
97042025-08-26T21:55:02.876ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
97052025-08-26T21:55:02.877ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
97062025-08-26T21:55:02.877ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
97072025-08-26T21:55:02.877ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
97082025-08-26T21:55:02.878ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
97092025-08-26T21:55:02.878ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
97102025-08-26T21:55:02.878ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
97112025-08-26T21:55:02.878ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
97122025-08-26T21:55:02.879ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
97132025-08-26T21:55:02.879ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
97142025-08-26T21:55:02.880ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
97152025-08-26T21:55:02.880ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
97162025-08-26T21:55:02.880ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
97172025-08-26T21:55:02.880ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
97182025-08-26T21:55:02.880ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
97192025-08-26T21:55:02.883ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
97202025-08-26T21:55:02.883ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
97212025-08-26T21:55:02.883ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
97222025-08-26T21:55:02.883ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
97232025-08-26T21:55:02.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
97242025-08-26T21:55:02.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
97252025-08-26T21:55:02.890ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
97262025-08-26T21:55:02.893ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
97272025-08-26T21:55:02.893ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
97282025-08-26T21:55:02.894ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
97292025-08-26T21:55:02.894ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
97302025-08-26T21:55:02.894ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
97312025-08-26T21:55:02.894ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
97322025-08-26T21:55:02.894ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
97332025-08-26T21:55:02.896ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
97342025-08-26T21:55:02.896ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
97352025-08-26T21:55:02.896ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
97362025-08-26T21:55:02.897ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1406, tv_nsec: 755681090 } })
97372025-08-26T21:55:02.897ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1205 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
97382025-08-26T21:55:02.898ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
97392025-08-26T21:55:02.899ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
97402025-08-26T21:55:02.899ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
97412025-08-26T21:55:02.899ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 30.753223ms iteration = 109 status = {"chicken_switches_updated":false}
97422025-08-26T21:55:02.900ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
97432025-08-26T21:55:02.901ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
97442025-08-26T21:55:02.907ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
97452025-08-26T21:55:02.907ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
97462025-08-26T21:55:02.907ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
97472025-08-26T21:55:02.936ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
97482025-08-26T21:55:02.939ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
97492025-08-26T21:55:02.939ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
97502025-08-26T21:55:02.939ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
97512025-08-26T21:55:02.939ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:170 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
97522025-08-26T21:55:02.939ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 79.453501ms iteration = 10 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
97532025-08-26T21:55:02.940ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
97542025-08-26T21:55:02.940ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
97552025-08-26T21:55:02.940ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
97562025-08-26T21:55:02.940ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
97572025-08-26T21:55:02.942ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
97582025-08-26T21:55:02.942ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
97592025-08-26T21:55:02.942ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
97602025-08-26T21:55:02.942ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
97612025-08-26T21:55:02.942ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
97622025-08-26T21:55:02.942ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:550 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
97632025-08-26T21:55:02.942ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 73.760332ms iteration = 19 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
97642025-08-26T21:55:02.942ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
97652025-08-26T21:55:02.943ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
97662025-08-26T21:55:02.944ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
97672025-08-26T21:55:02.944ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
97682025-08-26T21:55:02.944ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
97692025-08-26T21:55:02.944ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
97702025-08-26T21:55:02.945ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
97712025-08-26T21:55:02.945ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
97722025-08-26T21:55:02.946ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
97732025-08-26T21:55:02.946ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
97742025-08-26T21:55:02.946ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
97752025-08-26T21:55:02.946ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
97762025-08-26T21:55:02.946ZDEBGnexus (BackgroundTasks): activation complete background_task = crdb_node_id_collector elapsed = 86.386787ms iteration = 5 status = {"nsuccess":5}
97772025-08-26T21:55:02.946ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
97782025-08-26T21:55:02.947ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
97792025-08-26T21:55:02.948ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
97802025-08-26T21:55:02.948ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
97812025-08-26T21:55:02.948ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
97822025-08-26T21:55:02.948ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
97832025-08-26T21:55:02.949ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
97842025-08-26T21:55:02.951ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
97852025-08-26T21:55:02.951ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
97862025-08-26T21:55:02.951ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
97872025-08-26T21:55:02.951ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
97882025-08-26T21:55:02.951ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
97892025-08-26T21:55:02.951ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
97902025-08-26T21:55:02.951ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
97912025-08-26T21:55:02.951ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
97922025-08-26T21:55:02.951ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
97932025-08-26T21:55:02.951ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
97942025-08-26T21:55:02.951ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
97952025-08-26T21:55:02.952ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
97962025-08-26T21:55:02.952ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
97972025-08-26T21:55:02.952ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 83.579504ms iteration = 19 status = {"errors":[],"requests_created_ok":[]}
97982025-08-26T21:55:02.952ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
97992025-08-26T21:55:02.953ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
98002025-08-26T21:55:02.954ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 85.323453ms iteration = 19 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
98012025-08-26T21:55:02.954ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 85.869236ms iteration = 19 status = {"errors":[],"finish_invoked_ok":[]}
98022025-08-26T21:55:02.957ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
98032025-08-26T21:55:02.957ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
98042025-08-26T21:55:02.957ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
98052025-08-26T21:55:02.957ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
98062025-08-26T21:55:02.958ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
98072025-08-26T21:55:02.958ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 89.946788ms iteration = 19 status = {"errors":[],"garbage_collect_requested":[]}
98082025-08-26T21:55:02.961ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1213 zone_address = fd00:1122:3344:101::2
98092025-08-26T21:55:02.961ZDEBGnexus (MgsClient): client request background_task = nat_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
98102025-08-26T21:55:02.961ZDEBGnexus (MgsClient): client response background_task = nat_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "299a0347-ae3c-452f-9297-2ba50c4d2ef2", "content-length": "26", "date": "Tue, 26 Aug 2025 21:55:02 GMT"} })
98112025-08-26T21:55:02.961ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1216 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
98122025-08-26T21:55:02.964ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
98132025-08-26T21:55:02.980ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
98142025-08-26T21:55:02.994ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
98152025-08-26T21:55:02.995ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
98162025-08-26T21:55:02.995ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
98172025-08-26T21:55:02.998ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
98182025-08-26T21:55:02.998ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
98192025-08-26T21:55:02.998ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
98202025-08-26T21:55:02.998ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
98212025-08-26T21:55:02.998ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
98222025-08-26T21:55:02.998ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
98232025-08-26T21:55:02.998ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
98242025-08-26T21:55:02.999ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 130.141404ms iteration = 19 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
98252025-08-26T21:55:03.000ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 131.74859ms iteration = 55 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
98262025-08-26T21:55:03.000ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
98272025-08-26T21:55:03.000ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
98282025-08-26T21:55:03.000ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
98292025-08-26T21:55:03.001ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1213 zone_address = fd00:1122:3344:101::2
98302025-08-26T21:55:03.001ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
98312025-08-26T21:55:03.001ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "d992bd3e-dbd0-4c55-84cc-2954e02b5588", "content-length": "26", "date": "Tue, 26 Aug 2025 21:55:02 GMT"} })
98322025-08-26T21:55:03.001ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1216 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
98332025-08-26T21:55:03.003ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
98342025-08-26T21:55:03.004ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
98352025-08-26T21:55:03.004ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
98362025-08-26T21:55:03.004ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
98372025-08-26T21:55:03.004ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 8e720f02-dde8-4423-a6e3-91698373805a dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
98382025-08-26T21:55:03.004ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
98392025-08-26T21:55:03.004ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
98402025-08-26T21:55:03.004ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
98412025-08-26T21:55:03.004ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
98422025-08-26T21:55:03.005ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
98432025-08-26T21:55:03.005ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
98442025-08-26T21:55:03.005ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
98452025-08-26T21:55:03.005ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: dd0c655a-95c2-48fc-b4c3-9733ed4b0faa, lookup_type: ById(dd0c655a-95c2-48fc-b4c3-9733ed4b0faa) } result = Ok(())
98462025-08-26T21:55:03.005ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
98472025-08-26T21:55:03.006ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
98482025-08-26T21:55:03.006ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
98492025-08-26T21:55:03.007ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
98502025-08-26T21:55:03.007ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
98512025-08-26T21:55:03.007ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
98522025-08-26T21:55:03.007ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1213 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 zone_address = fd00:1122:3344:101::2
98532025-08-26T21:55:03.007ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
98542025-08-26T21:55:03.008ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "d7e67f09-9a90-486e-85d0-c4ad13ecfad9", "content-length": "26", "date": "Tue, 26 Aug 2025 21:55:02 GMT"} })
98552025-08-26T21:55:03.008ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1216 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
98562025-08-26T21:55:03.009ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
98572025-08-26T21:55:03.010ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
98582025-08-26T21:55:03.010ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
98592025-08-26T21:55:03.010ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
98602025-08-26T21:55:03.010ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 142.036481ms iteration = 19 status = 0
98612025-08-26T21:55:03.014ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 8e720f02-dde8-4423-a6e3-91698373805a dns_zone_name = control-plane.oxide.internal found_so_far = 59 version = 1
98622025-08-26T21:55:03.014ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
98632025-08-26T21:55:03.014ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2025-08-26 21:46:04.065217 UTC dns_group = internal generation = 1
98642025-08-26T21:55:03.014ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 181.700207ms iteration = 10 status = {"generation":1}
98652025-08-26T21:55:03.017ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
98662025-08-26T21:55:03.017ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
98672025-08-26T21:55:03.017ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
98682025-08-26T21:55:03.018ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
98692025-08-26T21:55:03.020ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
98702025-08-26T21:55:03.020ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:138 found = 0 not_ready_to_be_deleted = 0
98712025-08-26T21:55:03.020ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 160.399156ms iteration = 10 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
98722025-08-26T21:55:03.022ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
98732025-08-26T21:55:03.022ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 149.692475ms iteration = 55 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
98742025-08-26T21:55:03.022ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
98752025-08-26T21:55:03.022ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
98762025-08-26T21:55:03.022ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
98772025-08-26T21:55:03.027ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
98782025-08-26T21:55:03.027ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
98792025-08-26T21:55:03.028ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
98802025-08-26T21:55:03.037ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
98812025-08-26T21:55:03.038ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
98822025-08-26T21:55:03.054ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-08-26T21:45:59.866217Z, time_modified: 2025-08-26T21:45:59.866217Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 9396aa55-de6f-4d6f-96db-d28e1c76fa9c, db_silo: Silo { identity: SiloIdentity { id: 9396aa55-de6f-4d6f-96db-d28e1c76fa9c, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-08-26T21:46:05.245521Z, time_modified: 2025-08-26T21:46:05.245521Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "e8529d720e4b0de099753dcdb373abb870af04baacd7c7dd850f5254858081af" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 9396aa55-de6f-4d6f-96db-d28e1c76fa9c, db_silo: Silo { identity: SiloIdentity { id: 9396aa55-de6f-4d6f-96db-d28e1c76fa9c, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-08-26T21:46:05.245521Z, time_modified: 2025-08-26T21:46:05.245521Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "e8529d720e4b0de099753dcdb373abb870af04baacd7c7dd850f5254858081af" }] }) }
98832025-08-26T21:55:03.054ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 221.223047ms iteration = 10 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"9396aa55-de6f-4d6f-96db-d28e1c76fa9c","tls_certs":["e8529d720e4b0de099753dcdb373abb870af04baacd7c7dd850f5254858081af"]}},"default_endpoint":{"silo_id":"9396aa55-de6f-4d6f-96db-d28e1c76fa9c","tls_certs":["e8529d720e4b0de099753dcdb373abb870af04baacd7c7dd850f5254858081af"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
98842025-08-26T21:55:03.062ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
98852025-08-26T21:55:03.062ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
98862025-08-26T21:55:03.062ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 193.928851ms iteration = 19 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
98872025-08-26T21:55:03.068ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:407
98882025-08-26T21:55:03.068ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 199.342203ms iteration = 19 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
98892025-08-26T21:55:03.072ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 203.277133ms iteration = 19 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
98902025-08-26T21:55:03.074ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
98912025-08-26T21:55:03.074ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = dfb404ab-acb8-43de-afe8-b429af0c161d dns_zone_name = oxide.test found_so_far = 0 version = 2
98922025-08-26T21:55:03.074ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
98932025-08-26T21:55:03.074ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
98942025-08-26T21:55:03.076ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
98952025-08-26T21:55:03.079ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
98962025-08-26T21:55:03.079ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
98972025-08-26T21:55:03.079ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
98982025-08-26T21:55:03.079ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
98992025-08-26T21:55:03.079ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
99002025-08-26T21:55:03.081ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
99012025-08-26T21:55:03.081ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = dfb404ab-acb8-43de-afe8-b429af0c161d dns_zone_name = oxide.test found_so_far = 4 version = 2
99022025-08-26T21:55:03.081ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
99032025-08-26T21:55:03.081ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2025-08-26 21:46:05.296205 UTC dns_group = external generation = 2
99042025-08-26T21:55:03.081ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 248.959022ms iteration = 10 status = {"generation":2}
99052025-08-26T21:55:03.083ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
99062025-08-26T21:55:03.083ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
99072025-08-26T21:55:03.083ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
99082025-08-26T21:55:03.083ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
99092025-08-26T21:55:03.085ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
99102025-08-26T21:55:03.087ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
99112025-08-26T21:55:03.087ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
99122025-08-26T21:55:03.087ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
99132025-08-26T21:55:03.089ZINFOnexus (BackgroundTasks): instance reincarnation completed successfully background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:113 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0
99142025-08-26T21:55:03.089ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 228.47698ms iteration = 10 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
99152025-08-26T21:55:03.095ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 226.336752ms iteration = 20 status = {"physical_disks_added":0}
99162025-08-26T21:55:03.110ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
99172025-08-26T21:55:03.110ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
99182025-08-26T21:55:03.112ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 279.211354ms iteration = 19 status = null
99192025-08-26T21:55:03.131ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
99202025-08-26T21:55:03.132ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "e3d7e49c-136a-48b6-94b1-663d53a76751", "date": "Tue, 26 Aug 2025 21:55:03 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
99212025-08-26T21:55:03.132ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
99222025-08-26T21:55:03.135ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "ff934141-83e4-404a-98df-19df1adae175", "date": "Tue, 26 Aug 2025 21:55:03 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
99232025-08-26T21:55:03.135ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
99242025-08-26T21:55:03.137ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
99252025-08-26T21:55:03.137ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 305.044456ms iteration = 12 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
99262025-08-26T21:55:03.138ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 306.08587ms iteration = 19 status = {}
99272025-08-26T21:55:03.182ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
99282025-08-26T21:55:03.182ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "2ad26086-1519-4025-88f2-64da9dfe7468", "date": "Tue, 26 Aug 2025 21:55:03 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
99292025-08-26T21:55:03.182ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
99302025-08-26T21:55:03.184ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "6c4721b8-1c8e-4317-bf69-8310e85494b0", "date": "Tue, 26 Aug 2025 21:55:03 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
99312025-08-26T21:55:03.184ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
99322025-08-26T21:55:03.185ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "e69be239-eea3-418b-8350-86d5f91b642b", "date": "Tue, 26 Aug 2025 21:55:03 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
99332025-08-26T21:55:03.185ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
99342025-08-26T21:55:03.187ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
99352025-08-26T21:55:03.188ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 355.107255ms iteration = 12 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
99362025-08-26T21:55:03.195ZINFOnexus (BackgroundTasks): VPC route manager sled dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
99372025-08-26T21:55:03.195ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
99382025-08-26T21:55:03.195ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "33add5e1-a725-4342-939f-c636f7836512", "content-length": "372", "date": "Tue, 26 Aug 2025 21:55:03 GMT"} })
99392025-08-26T21:55:03.195ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
99402025-08-26T21:55:03.196ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
99412025-08-26T21:55:03.210ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
99422025-08-26T21:55:03.210ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
99432025-08-26T21:55:03.211ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "9903a778-0a22-4dfc-9e86-a404b9b0ecb7", "date": "Tue, 26 Aug 2025 21:55:03 GMT"} })
99442025-08-26T21:55:03.211ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
99452025-08-26T21:55:03.211ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
99462025-08-26T21:55:03.213ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
99472025-08-26T21:55:03.214ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
99482025-08-26T21:55:03.217ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
99492025-08-26T21:55:03.217ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
99502025-08-26T21:55:03.219ZINFOnexus (BackgroundTasks): VPC route manager sled dd0c655a-95c2-48fc-b4c3-9733ed4b0faa push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
99512025-08-26T21:55:03.219ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
99522025-08-26T21:55:03.220ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "fc5b623f-8eaa-4b07-a527-ab5ec5a8b8ca", "date": "Tue, 26 Aug 2025 21:55:03 GMT"} })
99532025-08-26T21:55:03.220ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
99542025-08-26T21:55:03.220ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "2c77ceb3-6890-449a-8290-2c7f358b530f", "content-length": "691", "date": "Tue, 26 Aug 2025 21:55:02 GMT"} })
99552025-08-26T21:55:03.220ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
99562025-08-26T21:55:03.220ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
99572025-08-26T21:55:03.220ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
99582025-08-26T21:55:03.222ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 353.11581ms iteration = 19 status = {}
99592025-08-26T21:55:03.223ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 354.402829ms iteration = 19 status = {}
99602025-08-26T21:55:03.241ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
99612025-08-26T21:55:03.242ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
99622025-08-26T21:55:03.244ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: d24c5c55-3a21-4a91-aeb4-45362e822df6, rack_id: e37a46f2-f789-4ca4-9841-f8c6dd705964, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(88d84f37-a4c8-4abc-af1f-27987b0d39d9) } rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0
99632025-08-26T21:55:03.244ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
99642025-08-26T21:55:03.244ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
99652025-08-26T21:55:03.256ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1832 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
99662025-08-26T21:55:03.256ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1859 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 88d84f37-a4c8-4abc-af1f-27987b0d39d9
99672025-08-26T21:55:03.257ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
99682025-08-26T21:55:03.257ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
99692025-08-26T21:55:03.257ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
99702025-08-26T21:55:03.257ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
99712025-08-26T21:55:03.257ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
99722025-08-26T21:55:03.257ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2006 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
99732025-08-26T21:55:03.257ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:408 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
99742025-08-26T21:55:03.257ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2042 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
99752025-08-26T21:55:03.257ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
99762025-08-26T21:55:03.257ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
99772025-08-26T21:55:03.257ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
99782025-08-26T21:55:03.259ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0
99792025-08-26T21:55:03.259ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
99802025-08-26T21:55:03.369ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "d8f355eb-75c6-4386-9bc9-340c954f409b", "date": "Tue, 26 Aug 2025 21:55:03 GMT"} })
99812025-08-26T21:55:03.369ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch = Switch0
99822025-08-26T21:55:03.369ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
99832025-08-26T21:55:03.369ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "74db8a61-e2cd-422b-8fe3-8658331c5e7d", "content-length": "682", "date": "Tue, 26 Aug 2025 21:55:03 GMT"} })
99842025-08-26T21:55:03.369ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
99852025-08-26T21:55:03.369ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
99862025-08-26T21:55:03.373ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
99872025-08-26T21:55:03.373ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
99882025-08-26T21:55:03.377ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
99892025-08-26T21:55:03.377ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
99902025-08-26T21:55:03.380ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
99912025-08-26T21:55:03.380ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
99922025-08-26T21:55:03.388ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 519.164623ms iteration = 19 status = {}
99932025-08-26T21:55:05.352ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 290 local_addr = [fd00:1122:3344:101::c]:42016 method = GET remote_addr = [fd00:1122:3344:101::d]:37550 req_id = 29bc94d4-f0a1-4f97-b658-bac3bf6e9f0b response_code = 200 uri = /5d0959e5-47a6-4bf3-a52b-57950b9d900d
99942025-08-26T21:55:07.869ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 110 reason = Timeout
99952025-08-26T21:55:07.869ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
99962025-08-26T21:55:07.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
99972025-08-26T21:55:07.871ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
99982025-08-26T21:55:07.874ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
99992025-08-26T21:55:07.874ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
100002025-08-26T21:55:07.874ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
100012025-08-26T21:55:07.875ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
100022025-08-26T21:55:07.875ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 6.521077ms iteration = 110 status = {"chicken_switches_updated":false}
100032025-08-26T21:55:12.868ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 56 reason = Timeout
100042025-08-26T21:55:12.868ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
100052025-08-26T21:55:12.869ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 111 reason = Timeout
100062025-08-26T21:55:12.869ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
100072025-08-26T21:55:12.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
100082025-08-26T21:55:12.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
100092025-08-26T21:55:12.871ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
100102025-08-26T21:55:12.871ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
100112025-08-26T21:55:12.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
100122025-08-26T21:55:12.872ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 56 reason = Timeout
100132025-08-26T21:55:12.872ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
100142025-08-26T21:55:12.872ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
100152025-08-26T21:55:12.873ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.533732ms iteration = 56 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
100162025-08-26T21:55:12.873ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
100172025-08-26T21:55:12.874ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
100182025-08-26T21:55:12.874ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
100192025-08-26T21:55:12.874ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
100202025-08-26T21:55:12.875ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
100212025-08-26T21:55:12.875ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 6.377784ms iteration = 111 status = {"chicken_switches_updated":false}
100222025-08-26T21:55:12.877ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
100232025-08-26T21:55:12.877ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
100242025-08-26T21:55:12.877ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
100252025-08-26T21:55:12.878ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
100262025-08-26T21:55:12.879ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
100272025-08-26T21:55:12.880ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
100282025-08-26T21:55:12.883ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
100292025-08-26T21:55:12.883ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
100302025-08-26T21:55:12.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
100312025-08-26T21:55:12.900ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
100322025-08-26T21:55:12.901ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 28.771528ms iteration = 56 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
100332025-08-26T21:55:15.352ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 300 local_addr = [fd00:1122:3344:101::c]:42016 method = GET remote_addr = [fd00:1122:3344:101::d]:37550 req_id = 7917ca29-f430-49bf-8035-4052cc2bdf0b response_code = 200 uri = /5d0959e5-47a6-4bf3-a52b-57950b9d900d
100342025-08-26T21:55:17.869ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 112 reason = Timeout
100352025-08-26T21:55:17.869ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
100362025-08-26T21:55:17.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
100372025-08-26T21:55:17.871ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
100382025-08-26T21:55:17.874ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
100392025-08-26T21:55:17.874ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
100402025-08-26T21:55:17.874ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
100412025-08-26T21:55:17.875ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
100422025-08-26T21:55:17.875ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 6.389564ms iteration = 112 status = {"chicken_switches_updated":false}
100432025-08-26T21:55:22.869ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 57 reason = Timeout
100442025-08-26T21:55:22.869ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
100452025-08-26T21:55:22.869ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 113 reason = Timeout
100462025-08-26T21:55:22.869ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
100472025-08-26T21:55:22.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
100482025-08-26T21:55:22.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
100492025-08-26T21:55:22.871ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
100502025-08-26T21:55:22.871ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 57 reason = Timeout
100512025-08-26T21:55:22.871ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
100522025-08-26T21:55:22.871ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
100532025-08-26T21:55:22.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
100542025-08-26T21:55:22.872ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
100552025-08-26T21:55:22.873ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
100562025-08-26T21:55:22.874ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
100572025-08-26T21:55:22.874ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
100582025-08-26T21:55:22.874ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
100592025-08-26T21:55:22.875ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
100602025-08-26T21:55:22.875ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 6.373423ms iteration = 113 status = {"chicken_switches_updated":false}
100612025-08-26T21:55:22.876ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
100622025-08-26T21:55:22.876ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
100632025-08-26T21:55:22.876ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
100642025-08-26T21:55:22.878ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
100652025-08-26T21:55:22.878ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
100662025-08-26T21:55:22.880ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
100672025-08-26T21:55:22.882ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
100682025-08-26T21:55:22.882ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
100692025-08-26T21:55:22.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
100702025-08-26T21:55:22.895ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
100712025-08-26T21:55:22.895ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 24.023581ms iteration = 57 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
100722025-08-26T21:55:22.908ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 39.195364ms iteration = 57 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
100732025-08-26T21:55:25.352ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 522 local_addr = [fd00:1122:3344:101::c]:42016 method = GET remote_addr = [fd00:1122:3344:101::d]:37550 req_id = a4df5cbb-8763-411d-af9b-9c4eb86ae212 response_code = 200 uri = /5d0959e5-47a6-4bf3-a52b-57950b9d900d
100742025-08-26T21:55:27.868ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 114 reason = Timeout
100752025-08-26T21:55:27.868ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
100762025-08-26T21:55:27.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
100772025-08-26T21:55:27.870ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
100782025-08-26T21:55:27.873ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
100792025-08-26T21:55:27.873ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
100802025-08-26T21:55:27.873ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
100812025-08-26T21:55:27.874ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
100822025-08-26T21:55:27.874ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 5.975715ms iteration = 114 status = {"chicken_switches_updated":false}
100832025-08-26T21:55:32.833ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 20 reason = Timeout
100842025-08-26T21:55:32.833ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 20 reason = Timeout
100852025-08-26T21:55:32.833ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 20 reason = Timeout
100862025-08-26T21:55:32.833ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
100872025-08-26T21:55:32.833ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
100882025-08-26T21:55:32.833ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
100892025-08-26T21:55:32.833ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
100902025-08-26T21:55:32.835ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1436, tv_nsec: 694314096 } })
100912025-08-26T21:55:32.835ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1436, tv_nsec: 694588493 } })
100922025-08-26T21:55:32.835ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1205
100932025-08-26T21:55:32.836ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1205
100942025-08-26T21:55:32.836ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.752965ms iteration = 20 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
100952025-08-26T21:55:32.868ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 20 reason = Timeout
100962025-08-26T21:55:32.868ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 20 reason = Timeout
100972025-08-26T21:55:32.868ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
100982025-08-26T21:55:32.868ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 20 reason = Timeout
100992025-08-26T21:55:32.868ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
101002025-08-26T21:55:32.868ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
101012025-08-26T21:55:32.869ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
101022025-08-26T21:55:32.869ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 21 reason = Timeout
101032025-08-26T21:55:32.869ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
101042025-08-26T21:55:32.869ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 20 reason = Timeout
101052025-08-26T21:55:32.869ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 20 reason = Timeout
101062025-08-26T21:55:32.869ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
101072025-08-26T21:55:32.869ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
101082025-08-26T21:55:32.869ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
101092025-08-26T21:55:32.869ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 20 reason = Timeout
101102025-08-26T21:55:32.869ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
101112025-08-26T21:55:32.869ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 20 reason = Timeout
101122025-08-26T21:55:32.869ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
101132025-08-26T21:55:32.869ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 20 reason = Timeout
101142025-08-26T21:55:32.869ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
101152025-08-26T21:55:32.869ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 20 reason = Timeout
101162025-08-26T21:55:32.869ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 20 reason = Timeout
101172025-08-26T21:55:32.869ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
101182025-08-26T21:55:32.869ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
101192025-08-26T21:55:32.869ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 20 reason = Timeout
101202025-08-26T21:55:32.869ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 12.67\u{b5}s iteration = 20 status = {"disabled":true,"errors":[],"sps":[]}
101212025-08-26T21:55:32.869ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 58 reason = Timeout
101222025-08-26T21:55:32.869ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 115 reason = Timeout
101232025-08-26T21:55:32.869ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
101242025-08-26T21:55:32.869ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 20 reason = Timeout
101252025-08-26T21:55:32.869ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
101262025-08-26T21:55:32.869ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 20 reason = Timeout
101272025-08-26T21:55:32.869ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
101282025-08-26T21:55:32.869ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
101292025-08-26T21:55:32.869ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 20 reason = Timeout
101302025-08-26T21:55:32.869ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
101312025-08-26T21:55:32.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
101322025-08-26T21:55:32.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
101332025-08-26T21:55:32.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
101342025-08-26T21:55:32.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
101352025-08-26T21:55:32.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
101362025-08-26T21:55:32.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
101372025-08-26T21:55:32.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
101382025-08-26T21:55:32.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
101392025-08-26T21:55:32.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
101402025-08-26T21:55:32.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
101412025-08-26T21:55:32.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
101422025-08-26T21:55:32.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
101432025-08-26T21:55:32.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
101442025-08-26T21:55:32.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
101452025-08-26T21:55:32.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
101462025-08-26T21:55:32.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
101472025-08-26T21:55:32.871ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
101482025-08-26T21:55:32.871ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
101492025-08-26T21:55:32.871ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
101502025-08-26T21:55:32.871ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
101512025-08-26T21:55:32.871ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
101522025-08-26T21:55:32.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
101532025-08-26T21:55:32.871ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
101542025-08-26T21:55:32.871ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
101552025-08-26T21:55:32.871ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
101562025-08-26T21:55:32.871ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
101572025-08-26T21:55:32.872ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 58 reason = Timeout
101582025-08-26T21:55:32.872ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
101592025-08-26T21:55:32.872ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
101602025-08-26T21:55:32.873ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
101612025-08-26T21:55:32.873ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
101622025-08-26T21:55:32.873ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
101632025-08-26T21:55:32.873ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
101642025-08-26T21:55:32.874ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
101652025-08-26T21:55:32.876ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
101662025-08-26T21:55:32.876ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
101672025-08-26T21:55:32.876ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
101682025-08-26T21:55:32.876ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
101692025-08-26T21:55:32.876ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
101702025-08-26T21:55:32.877ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
101712025-08-26T21:55:32.877ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
101722025-08-26T21:55:32.877ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
101732025-08-26T21:55:32.877ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
101742025-08-26T21:55:32.877ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
101752025-08-26T21:55:32.877ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
101762025-08-26T21:55:32.877ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
101772025-08-26T21:55:32.877ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
101782025-08-26T21:55:32.877ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
101792025-08-26T21:55:32.877ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
101802025-08-26T21:55:32.877ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
101812025-08-26T21:55:32.877ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
101822025-08-26T21:55:32.877ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
101832025-08-26T21:55:32.879ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
101842025-08-26T21:55:32.879ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
101852025-08-26T21:55:32.879ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
101862025-08-26T21:55:32.879ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
101872025-08-26T21:55:32.879ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
101882025-08-26T21:55:32.879ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 10.787914ms iteration = 115 status = {"chicken_switches_updated":false}
101892025-08-26T21:55:32.879ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
101902025-08-26T21:55:32.879ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
101912025-08-26T21:55:32.879ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1436, tv_nsec: 738680836 } })
101922025-08-26T21:55:32.880ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
101932025-08-26T21:55:32.880ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1205 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
101942025-08-26T21:55:32.881ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
101952025-08-26T21:55:32.881ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
101962025-08-26T21:55:32.889ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
101972025-08-26T21:55:32.893ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
101982025-08-26T21:55:32.893ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
101992025-08-26T21:55:32.893ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
102002025-08-26T21:55:32.909ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1213 zone_address = fd00:1122:3344:101::2
102012025-08-26T21:55:32.909ZDEBGnexus (MgsClient): client request background_task = nat_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
102022025-08-26T21:55:32.910ZDEBGnexus (MgsClient): client response background_task = nat_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "8f949d6a-19e0-48ec-85b3-0c0eb6376aad", "content-length": "26", "date": "Tue, 26 Aug 2025 21:55:32 GMT"} })
102032025-08-26T21:55:32.910ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1216 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
102042025-08-26T21:55:32.911ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
102052025-08-26T21:55:32.916ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1213 zone_address = fd00:1122:3344:101::2
102062025-08-26T21:55:32.916ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
102072025-08-26T21:55:32.916ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "c5ef8995-b68d-47bd-af9b-1e70a6aa4eb5", "content-length": "26", "date": "Tue, 26 Aug 2025 21:55:32 GMT"} })
102082025-08-26T21:55:32.916ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1216 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
102092025-08-26T21:55:32.917ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
102102025-08-26T21:55:32.918ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
102112025-08-26T21:55:32.918ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 45.863664ms iteration = 58 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
102122025-08-26T21:55:32.918ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 49.558477ms iteration = 20 status = {"errors":[],"garbage_collect_requested":[]}
102132025-08-26T21:55:32.918ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
102142025-08-26T21:55:32.919ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
102152025-08-26T21:55:32.921ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
102162025-08-26T21:55:32.921ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
102172025-08-26T21:55:32.922ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 53.357062ms iteration = 20 status = {"errors":[],"finish_invoked_ok":[]}
102182025-08-26T21:55:32.923ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
102192025-08-26T21:55:32.923ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
102202025-08-26T21:55:32.923ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
102212025-08-26T21:55:32.925ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
102222025-08-26T21:55:32.925ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
102232025-08-26T21:55:32.925ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 56.30097ms iteration = 20 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
102242025-08-26T21:55:32.926ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: dd0c655a-95c2-48fc-b4c3-9733ed4b0faa, lookup_type: ById(dd0c655a-95c2-48fc-b4c3-9733ed4b0faa) } result = Ok(())
102252025-08-26T21:55:32.926ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
102262025-08-26T21:55:32.926ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
102272025-08-26T21:55:32.928ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 59.220624ms iteration = 20 status = {"errors":[],"requests_created_ok":[]}
102282025-08-26T21:55:32.928ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 59.321018ms iteration = 20 status = 0
102292025-08-26T21:55:32.930ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
102302025-08-26T21:55:32.931ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
102312025-08-26T21:55:32.931ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
102322025-08-26T21:55:32.931ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
102332025-08-26T21:55:32.932ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
102342025-08-26T21:55:32.932ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
102352025-08-26T21:55:32.932ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
102362025-08-26T21:55:32.932ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
102372025-08-26T21:55:32.933ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 64.428722ms iteration = 20 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
102382025-08-26T21:55:32.933ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 64.480784ms iteration = 58 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
102392025-08-26T21:55:32.934ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
102402025-08-26T21:55:32.934ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
102412025-08-26T21:55:32.934ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:550 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
102422025-08-26T21:55:32.934ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 65.455825ms iteration = 20 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
102432025-08-26T21:55:32.934ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
102442025-08-26T21:55:32.936ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 67.146613ms iteration = 20 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
102452025-08-26T21:55:32.936ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
102462025-08-26T21:55:32.936ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
102472025-08-26T21:55:32.936ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
102482025-08-26T21:55:32.936ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
102492025-08-26T21:55:32.936ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
102502025-08-26T21:55:32.937ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
102512025-08-26T21:55:32.940ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
102522025-08-26T21:55:32.940ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
102532025-08-26T21:55:32.940ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
102542025-08-26T21:55:32.940ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
102552025-08-26T21:55:32.942ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
102562025-08-26T21:55:32.942ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
102572025-08-26T21:55:32.942ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
102582025-08-26T21:55:32.965ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
102592025-08-26T21:55:32.965ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
102602025-08-26T21:55:32.977ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
102612025-08-26T21:55:32.977ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
102622025-08-26T21:55:32.978ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1213 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 zone_address = fd00:1122:3344:101::2
102632025-08-26T21:55:32.978ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
102642025-08-26T21:55:32.978ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "1ea8ac50-170e-4782-a2fa-0bbe22ffeb68", "content-length": "26", "date": "Tue, 26 Aug 2025 21:55:32 GMT"} })
102652025-08-26T21:55:32.978ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1216 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
102662025-08-26T21:55:32.979ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:407
102672025-08-26T21:55:32.979ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 110.92649ms iteration = 20 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
102682025-08-26T21:55:32.980ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
102692025-08-26T21:55:32.998ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 129.933769ms iteration = 21 status = {"physical_disks_added":0}
102702025-08-26T21:55:33.005ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
102712025-08-26T21:55:33.005ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
102722025-08-26T21:55:33.008ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 175.021895ms iteration = 20 status = null
102732025-08-26T21:55:33.011ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 142.637816ms iteration = 20 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
102742025-08-26T21:55:33.035ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 202.649398ms iteration = 20 status = {}
102752025-08-26T21:55:33.040ZINFOnexus (BackgroundTasks): VPC route manager sled dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
102762025-08-26T21:55:33.040ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
102772025-08-26T21:55:33.040ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "1bcc59d0-6516-4709-aee8-7af01737140d", "content-length": "372", "date": "Tue, 26 Aug 2025 21:55:33 GMT"} })
102782025-08-26T21:55:33.040ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
102792025-08-26T21:55:33.041ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
102802025-08-26T21:55:33.098ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
102812025-08-26T21:55:33.098ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
102822025-08-26T21:55:33.098ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "3b68246b-3bb3-4145-9682-63d20f74f268", "date": "Tue, 26 Aug 2025 21:55:33 GMT"} })
102832025-08-26T21:55:33.098ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
102842025-08-26T21:55:33.099ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
102852025-08-26T21:55:33.100ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
102862025-08-26T21:55:33.101ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
102872025-08-26T21:55:33.101ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
102882025-08-26T21:55:33.102ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
102892025-08-26T21:55:33.102ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
102902025-08-26T21:55:33.104ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
102912025-08-26T21:55:33.104ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
102922025-08-26T21:55:33.104ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
102932025-08-26T21:55:33.104ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
102942025-08-26T21:55:33.104ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
102952025-08-26T21:55:33.106ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
102962025-08-26T21:55:33.106ZINFOnexus (BackgroundTasks): VPC route manager sled dd0c655a-95c2-48fc-b4c3-9733ed4b0faa push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
102972025-08-26T21:55:33.106ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
102982025-08-26T21:55:33.107ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "c9f94ba3-d893-4eb4-b5e9-e0ca5b64e3f4", "date": "Tue, 26 Aug 2025 21:55:33 GMT"} })
102992025-08-26T21:55:33.108ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
103002025-08-26T21:55:33.108ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
103012025-08-26T21:55:33.108ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
103022025-08-26T21:55:33.108ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 239.860127ms iteration = 20 status = {}
103032025-08-26T21:55:33.136ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
103042025-08-26T21:55:33.136ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
103052025-08-26T21:55:33.138ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: d24c5c55-3a21-4a91-aeb4-45362e822df6, rack_id: e37a46f2-f789-4ca4-9841-f8c6dd705964, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(88d84f37-a4c8-4abc-af1f-27987b0d39d9) } rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0
103062025-08-26T21:55:33.139ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
103072025-08-26T21:55:33.139ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
103082025-08-26T21:55:33.150ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1832 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
103092025-08-26T21:55:33.150ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1859 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 88d84f37-a4c8-4abc-af1f-27987b0d39d9
103102025-08-26T21:55:33.151ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
103112025-08-26T21:55:33.151ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
103122025-08-26T21:55:33.151ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
103132025-08-26T21:55:33.151ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
103142025-08-26T21:55:33.151ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
103152025-08-26T21:55:33.151ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2006 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
103162025-08-26T21:55:33.151ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:408 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
103172025-08-26T21:55:33.151ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2042 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
103182025-08-26T21:55:33.151ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
103192025-08-26T21:55:33.151ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
103202025-08-26T21:55:33.151ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
103212025-08-26T21:55:33.154ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0
103222025-08-26T21:55:33.154ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
103232025-08-26T21:55:33.154ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
103242025-08-26T21:55:33.161ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "3f7f8614-ad5a-49d5-9ce5-b38f8c0516fa", "content-length": "691", "date": "Tue, 26 Aug 2025 21:55:33 GMT"} })
103252025-08-26T21:55:33.161ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
103262025-08-26T21:55:33.161ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
103272025-08-26T21:55:33.161ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
103282025-08-26T21:55:33.163ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 294.273173ms iteration = 20 status = {}
103292025-08-26T21:55:33.266ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "d2862a91-fd54-4614-a512-793714dcb22d", "date": "Tue, 26 Aug 2025 21:55:33 GMT"} })
103302025-08-26T21:55:33.266ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch = Switch0
103312025-08-26T21:55:33.266ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
103322025-08-26T21:55:33.266ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "bc126876-825a-4930-9639-32f09dd982e0", "content-length": "682", "date": "Tue, 26 Aug 2025 21:55:33 GMT"} })
103332025-08-26T21:55:33.266ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
103342025-08-26T21:55:33.267ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
103352025-08-26T21:55:33.270ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
103362025-08-26T21:55:33.271ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
103372025-08-26T21:55:33.274ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
103382025-08-26T21:55:33.274ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
103392025-08-26T21:55:33.276ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
103402025-08-26T21:55:33.276ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
103412025-08-26T21:55:33.283ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 414.718948ms iteration = 20 status = {}
103422025-08-26T21:55:35.352ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 310 local_addr = [fd00:1122:3344:101::c]:42016 method = GET remote_addr = [fd00:1122:3344:101::d]:37550 req_id = ef54f5db-01aa-4e00-9406-d80f1c1820d3 response_code = 200 uri = /5d0959e5-47a6-4bf3-a52b-57950b9d900d
103432025-08-26T21:55:37.869ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 116 reason = Timeout
103442025-08-26T21:55:37.869ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
103452025-08-26T21:55:37.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
103462025-08-26T21:55:37.871ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
103472025-08-26T21:55:37.874ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
103482025-08-26T21:55:37.874ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
103492025-08-26T21:55:37.874ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
103502025-08-26T21:55:37.876ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
103512025-08-26T21:55:37.876ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.018619ms iteration = 116 status = {"chicken_switches_updated":false}
103522025-08-26T21:55:42.868ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 59 reason = Timeout
103532025-08-26T21:55:42.868ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
103542025-08-26T21:55:42.868ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 117 reason = Timeout
103552025-08-26T21:55:42.868ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
103562025-08-26T21:55:42.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
103572025-08-26T21:55:42.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
103582025-08-26T21:55:42.870ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
103592025-08-26T21:55:42.871ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
103602025-08-26T21:55:42.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
103612025-08-26T21:55:42.872ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 59 reason = Timeout
103622025-08-26T21:55:42.872ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
103632025-08-26T21:55:42.872ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.867567ms iteration = 59 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
103642025-08-26T21:55:42.872ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
103652025-08-26T21:55:42.873ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
103662025-08-26T21:55:42.873ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
103672025-08-26T21:55:42.873ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
103682025-08-26T21:55:42.874ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
103692025-08-26T21:55:42.875ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
103702025-08-26T21:55:42.875ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 6.67171ms iteration = 117 status = {"chicken_switches_updated":false}
103712025-08-26T21:55:42.876ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
103722025-08-26T21:55:42.876ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
103732025-08-26T21:55:42.876ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
103742025-08-26T21:55:42.878ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
103752025-08-26T21:55:42.878ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
103762025-08-26T21:55:42.879ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
103772025-08-26T21:55:42.882ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
103782025-08-26T21:55:42.882ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
103792025-08-26T21:55:42.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
103802025-08-26T21:55:42.897ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
103812025-08-26T21:55:42.897ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 25.30737ms iteration = 59 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
103822025-08-26T21:55:45.352ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 279 local_addr = [fd00:1122:3344:101::c]:42016 method = GET remote_addr = [fd00:1122:3344:101::d]:37550 req_id = a5271bd9-feca-4eee-9ebf-a18725b1e750 response_code = 200 uri = /5d0959e5-47a6-4bf3-a52b-57950b9d900d
103832025-08-26T21:55:47.869ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 118 reason = Timeout
103842025-08-26T21:55:47.869ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
103852025-08-26T21:55:47.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
103862025-08-26T21:55:47.872ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
103872025-08-26T21:55:47.875ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
103882025-08-26T21:55:47.875ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
103892025-08-26T21:55:47.875ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
103902025-08-26T21:55:47.877ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
103912025-08-26T21:55:47.877ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.678693ms iteration = 118 status = {"chicken_switches_updated":false}
103922025-08-26T21:55:52.869ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 60 reason = Timeout
103932025-08-26T21:55:52.869ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
103942025-08-26T21:55:52.869ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 119 reason = Timeout
103952025-08-26T21:55:52.869ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
103962025-08-26T21:55:52.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
103972025-08-26T21:55:52.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
103982025-08-26T21:55:52.871ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
103992025-08-26T21:55:52.871ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 60 reason = Timeout
104002025-08-26T21:55:52.871ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
104012025-08-26T21:55:52.871ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
104022025-08-26T21:55:52.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
104032025-08-26T21:55:52.872ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
104042025-08-26T21:55:52.873ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
104052025-08-26T21:55:52.873ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.248225ms iteration = 60 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
104062025-08-26T21:55:52.875ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
104072025-08-26T21:55:52.875ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
104082025-08-26T21:55:52.875ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
104092025-08-26T21:55:52.876ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
104102025-08-26T21:55:52.876ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
104112025-08-26T21:55:52.877ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
104122025-08-26T21:55:52.877ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
104132025-08-26T21:55:52.877ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.717284ms iteration = 119 status = {"chicken_switches_updated":false}
104142025-08-26T21:55:52.878ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
104152025-08-26T21:55:52.878ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
104162025-08-26T21:55:52.879ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
104172025-08-26T21:55:52.882ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
104182025-08-26T21:55:52.882ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
104192025-08-26T21:55:52.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
104202025-08-26T21:55:52.894ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
104212025-08-26T21:55:52.894ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 23.279345ms iteration = 60 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
104222025-08-26T21:55:55.352ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 346 local_addr = [fd00:1122:3344:101::c]:42016 method = GET remote_addr = [fd00:1122:3344:101::d]:37550 req_id = c3cf0fc8-4bb6-4ed8-a05d-49bee290419a response_code = 200 uri = /5d0959e5-47a6-4bf3-a52b-57950b9d900d
104232025-08-26T21:55:57.868ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 120 reason = Timeout
104242025-08-26T21:55:57.869ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
104252025-08-26T21:55:57.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
104262025-08-26T21:55:57.871ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
104272025-08-26T21:55:57.874ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
104282025-08-26T21:55:57.874ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
104292025-08-26T21:55:57.874ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
104302025-08-26T21:55:57.876ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
104312025-08-26T21:55:57.876ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.450368ms iteration = 120 status = {"chicken_switches_updated":false}
104322025-08-26T21:56:00.345ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:1025 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::d]:61737
104332025-08-26T21:56:00.345ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:61737 req_id = d2797615-82f6-49ea-af25-664cff5d9183 roles = RoleSet { roles: {} } uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?sort_by=id_ascending
104342025-08-26T21:56:00.346ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:61737 req_id = d2797615-82f6-49ea-af25-664cff5d9183 resource = Database result = Ok(()) uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?sort_by=id_ascending
104352025-08-26T21:56:00.348ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 3453 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:61737 req_id = d2797615-82f6-49ea-af25-664cff5d9183 response_code = 200 uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?sort_by=id_ascending
104362025-08-26T21:56:00.349ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:61737 req_id = 6252222a-cbf5-4fa8-b0c3-577a04b44319 roles = RoleSet { roles: {} } uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMzhjYTg4YS1lZGE1LTQwZGEtOGE4MS1hN2UzMzdhMmM4ZmIifX0%3D
104372025-08-26T21:56:00.349ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:61737 req_id = 6252222a-cbf5-4fa8-b0c3-577a04b44319 resource = Database result = Ok(()) uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMzhjYTg4YS1lZGE1LTQwZGEtOGE4MS1hN2UzMzdhMmM4ZmIifX0%3D
104382025-08-26T21:56:00.351ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 2442 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:61737 req_id = 6252222a-cbf5-4fa8-b0c3-577a04b44319 response_code = 200 uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMzhjYTg4YS1lZGE1LTQwZGEtOGE4MS1hN2UzMzdhMmM4ZmIifX0%3D
104392025-08-26T21:56:02.832ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 11 reason = Timeout
104402025-08-26T21:56:02.832ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 11 reason = Timeout
104412025-08-26T21:56:02.832ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
104422025-08-26T21:56:02.832ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
104432025-08-26T21:56:02.832ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 21 reason = Timeout
104442025-08-26T21:56:02.832ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 21 reason = Timeout
104452025-08-26T21:56:02.832ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
104462025-08-26T21:56:02.832ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 13 reason = Timeout
104472025-08-26T21:56:02.832ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 21 reason = Timeout
104482025-08-26T21:56:02.832ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
104492025-08-26T21:56:02.832ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 11 reason = Timeout
104502025-08-26T21:56:02.832ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 13 reason = Timeout
104512025-08-26T21:56:02.832ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-08-26 21:46:02.832871308 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
104522025-08-26T21:56:02.832ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
104532025-08-26T21:56:02.832ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 11 reason = Timeout
104542025-08-26T21:56:02.832ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 11 reason = Timeout
104552025-08-26T21:56:02.832ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 11 reason = Timeout
104562025-08-26T21:56:02.832ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
104572025-08-26T21:56:02.832ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
104582025-08-26T21:56:02.832ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
104592025-08-26T21:56:02.833ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
104602025-08-26T21:56:02.833ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
104612025-08-26T21:56:02.833ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
104622025-08-26T21:56:02.833ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
104632025-08-26T21:56:02.833ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
104642025-08-26T21:56:02.833ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
104652025-08-26T21:56:02.833ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("60abf731-31e1-4111-a3e2-65da89eaed87.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("84c849eb-0949-41d9-ad02-9a5509144564.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("84c849eb-0949-41d9-ad02-9a5509144564.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::8)) }], valid_until: Instant { tv_sec: 1466, tv_nsec: 692175408 } })
104662025-08-26T21:56:02.833ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("852fd1a3-de9a-47e8-a667-5e8dab880ad3.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("c68f1501-294c-4874-92ca-8fc0546a5870.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("d741a72f-8745-4723-9e80-e1e72987266b.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("d741a72f-8745-4723-9e80-e1e72987266b.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:3::1)) }], valid_until: Instant { tv_sec: 1466, tv_nsec: 692226840 } })
104672025-08-26T21:56:02.833ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
104682025-08-26T21:56:02.833ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.223757ms iteration = 11 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
104692025-08-26T21:56:02.834ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } dns_group = internal
104702025-08-26T21:56:02.834ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.377342ms iteration = 11 status = {"addresses":["[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:3::1]:5353"]}
104712025-08-26T21:56:02.834ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
104722025-08-26T21:56:02.834ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
104732025-08-26T21:56:02.834ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
104742025-08-26T21:56:02.834ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
104752025-08-26T21:56:02.835ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 12 reason = Timeout
104762025-08-26T21:56:02.835ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:91 reason = blueprint disabled target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8
104772025-08-26T21:56:02.835ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 15.601\u{b5}s iteration = 12 status = {"enabled":false,"target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8"}
104782025-08-26T21:56:02.836ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.269414ms iteration = 21 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
104792025-08-26T21:56:02.836ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1466, tv_nsec: 694817628 } })
104802025-08-26T21:56:02.836ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1466, tv_nsec: 694947431 } })
104812025-08-26T21:56:02.836ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-08-26 21:46:02.832871308 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
104822025-08-26T21:56:02.836ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 3.442397ms iteration = 11 status = {"expiration":"2025-08-26T21:46:02.832871308Z","pruned":[]}
104832025-08-26T21:56:02.836ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1205
104842025-08-26T21:56:02.836ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1205
104852025-08-26T21:56:02.836ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
104862025-08-26T21:56:02.836ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
104872025-08-26T21:56:02.841ZDEBGnexus (BackgroundTasks): activating background_task = inventory_collection iteration = 2 reason = Timeout
104882025-08-26T21:56:02.841ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
104892025-08-26T21:56:02.841ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
104902025-08-26T21:56:02.841ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
104912025-08-26T21:56:02.842ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
104922025-08-26T21:56:02.842ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
104932025-08-26T21:56:02.842ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
104942025-08-26T21:56:02.842ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
104952025-08-26T21:56:02.843ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
104962025-08-26T21:56:02.843ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
104972025-08-26T21:56:02.843ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
104982025-08-26T21:56:02.845ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
104992025-08-26T21:56:02.846ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
105002025-08-26T21:56:02.846ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
105012025-08-26T21:56:02.846ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
105022025-08-26T21:56:02.850ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
105032025-08-26T21:56:02.850ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
105042025-08-26T21:56:02.850ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
105052025-08-26T21:56:02.859ZDEBGnexus (BackgroundTasks): inventory_prune_one: eligible for removal background_task = inventory_collection candidates = [(98fcdfec-5acc-4b21-be95-18fc820aa3d1, 0), (f37c741e-e5eb-4be5-8e12-bbe92da18b97, 0), (22ce40dd-e0b9-460a-8151-60c0f8e6fa4f, 0), (08443897-9b9f-4f71-92e4-66f0b6ae2a76, 0)] collection_id = 98fcdfec-5acc-4b21-be95-18fc820aa3d1
105062025-08-26T21:56:02.859ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
105072025-08-26T21:56:02.859ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
105082025-08-26T21:56:02.860ZDEBGnexus (BackgroundTasks): activating background_task = saga_recovery iteration = 2 reason = Timeout
105092025-08-26T21:56:02.860ZINFOnexus (SagaQuiesceHandle): saga recovery pass starting file = nexus/types/src/quiesce.rs:319
105102025-08-26T21:56:02.860ZDEBGnexus (SagaRecoverer): listing candidate sagas for recovery
105112025-08-26T21:56:02.860ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 13 reason = Timeout
105122025-08-26T21:56:02.860ZDEBGnexus (SagaRecoverer): roles roles = RoleSet { roles: {} }
105132025-08-26T21:56:02.860ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_rendezvous iteration = 4 reason = Timeout
105142025-08-26T21:56:02.860ZDEBGnexus (BackgroundTasks): blueprint planning disabled, doing nothing background_task = blueprint_planner
105152025-08-26T21:56:02.860ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 15.75\u{b5}s iteration = 13 status = "Disabled"
105162025-08-26T21:56:02.860ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
105172025-08-26T21:56:02.860ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 11 reason = Timeout
105182025-08-26T21:56:02.860ZDEBGnexus (BackgroundTasks): activating background_task = service_firewall_rule_propagation iteration = 3 reason = Timeout
105192025-08-26T21:56:02.860ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:53
105202025-08-26T21:56:02.860ZDEBGnexus (service-firewall-rule-progator): starting background task for service firewall rule propagation background_task = service_firewall_rule_propagation
105212025-08-26T21:56:02.860ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 11 reason = Timeout
105222025-08-26T21:56:02.860ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
105232025-08-26T21:56:02.860ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
105242025-08-26T21:56:02.860ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
105252025-08-26T21:56:02.860ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 11 reason = Timeout
105262025-08-26T21:56:02.860ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
105272025-08-26T21:56:02.860ZDEBGnexus (BackgroundTasks): activating background_task = tuf_artifact_replication iteration = 3 reason = Timeout
105282025-08-26T21:56:02.860ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
105292025-08-26T21:56:02.860ZDEBGnexus (BackgroundTasks): activating background_task = alert_dispatcher iteration = 11 reason = Timeout
105302025-08-26T21:56:02.860ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 11 reason = Timeout
105312025-08-26T21:56:02.860ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
105322025-08-26T21:56:02.860ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
105332025-08-26T21:56:02.860ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
105342025-08-26T21:56:02.860ZDEBGnexus (BackgroundTasks): activating background_task = webhook_deliverator iteration = 11 reason = Timeout
105352025-08-26T21:56:02.860ZDEBGnexus (BackgroundTasks): roles background_task = webhook_deliverator roles = RoleSet { roles: {} }
105362025-08-26T21:56:02.860ZDEBGnexus (SagaRecoverer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000003a8a, .. }) resource = Database result = Ok(())
105372025-08-26T21:56:02.860ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
105382025-08-26T21:56:02.860ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
105392025-08-26T21:56:02.860ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
105402025-08-26T21:56:02.860ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
105412025-08-26T21:56:02.860ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
105422025-08-26T21:56:02.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = alert_dispatcher resource = Database result = Ok(())
105432025-08-26T21:56:02.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
105442025-08-26T21:56:02.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = webhook_deliverator resource = Database result = Ok(())
105452025-08-26T21:56:02.862ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
105462025-08-26T21:56:02.862ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
105472025-08-26T21:56:02.862ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
105482025-08-26T21:56:02.863ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
105492025-08-26T21:56:02.863ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
105502025-08-26T21:56:02.863ZINFOnexus (BackgroundTasks): all webhook delivery tasks completed background_task = webhook_deliverator file = nexus/src/app/background/tasks/webhook_deliverator.rs:204 num_receivers = 0
105512025-08-26T21:56:02.863ZDEBGnexus (BackgroundTasks): activation complete background_task = webhook_deliverator elapsed = 3.126751ms iteration = 11 status = {"by_rx":{},"error":null}
105522025-08-26T21:56:02.863ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
105532025-08-26T21:56:02.863ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:170 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
105542025-08-26T21:56:02.863ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 3.567331ms iteration = 11 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
105552025-08-26T21:56:02.863ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
105562025-08-26T21:56:02.863ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 3.306674ms iteration = 11 status = {"errors":[],"found_port_ok":[]}
105572025-08-26T21:56:02.864ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
105582025-08-26T21:56:02.865ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
105592025-08-26T21:56:02.865ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
105602025-08-26T21:56:02.865ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = alert_dispatcher resource = Database result = Ok(())
105612025-08-26T21:56:02.865ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
105622025-08-26T21:56:02.867ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Fleet result = Ok(())
105632025-08-26T21:56:02.867ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
105642025-08-26T21:56:02.867ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
105652025-08-26T21:56:02.867ZDEBGnexus (BackgroundTasks): activation complete background_task = alert_dispatcher elapsed = 7.330145ms iteration = 11 status = {"dispatched":[],"errors":[],"glob_version":"183.0.0","globs_reprocessed":{},"no_receivers":[]}
105662025-08-26T21:56:02.868ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
105672025-08-26T21:56:02.868ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
105682025-08-26T21:56:02.868ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Inventory result = Ok(())
105692025-08-26T21:56:02.868ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
105702025-08-26T21:56:02.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
105712025-08-26T21:56:02.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
105722025-08-26T21:56:02.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
105732025-08-26T21:56:02.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
105742025-08-26T21:56:02.868ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 21 reason = Timeout
105752025-08-26T21:56:02.868ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 21 reason = Timeout
105762025-08-26T21:56:02.868ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 17.53\u{b5}s iteration = 21 status = {"disabled":true,"errors":[],"sps":[]}
105772025-08-26T21:56:02.868ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
105782025-08-26T21:56:02.868ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 21 reason = Timeout
105792025-08-26T21:56:02.868ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 21 reason = Timeout
105802025-08-26T21:56:02.868ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
105812025-08-26T21:56:02.868ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
105822025-08-26T21:56:02.868ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 21 reason = Timeout
105832025-08-26T21:56:02.868ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
105842025-08-26T21:56:02.868ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 21 reason = Timeout
105852025-08-26T21:56:02.868ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 21 reason = Timeout
105862025-08-26T21:56:02.868ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
105872025-08-26T21:56:02.868ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
105882025-08-26T21:56:02.868ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 21 reason = Timeout
105892025-08-26T21:56:02.868ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
105902025-08-26T21:56:02.868ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 21 reason = Timeout
105912025-08-26T21:56:02.868ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
105922025-08-26T21:56:02.868ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 21 reason = Timeout
105932025-08-26T21:56:02.868ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
105942025-08-26T21:56:02.868ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
105952025-08-26T21:56:02.868ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 21 reason = Timeout
105962025-08-26T21:56:02.868ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 21 reason = Timeout
105972025-08-26T21:56:02.868ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
105982025-08-26T21:56:02.868ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
105992025-08-26T21:56:02.868ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 61 reason = Timeout
106002025-08-26T21:56:02.868ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 22 reason = Timeout
106012025-08-26T21:56:02.869ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
106022025-08-26T21:56:02.869ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
106032025-08-26T21:56:02.869ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 121 reason = Timeout
106042025-08-26T21:56:02.869ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 21 reason = Timeout
106052025-08-26T21:56:02.869ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
106062025-08-26T21:56:02.869ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
106072025-08-26T21:56:02.869ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
106082025-08-26T21:56:02.869ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 21 reason = Timeout
106092025-08-26T21:56:02.869ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
106102025-08-26T21:56:02.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
106112025-08-26T21:56:02.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
106122025-08-26T21:56:02.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
106132025-08-26T21:56:02.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
106142025-08-26T21:56:02.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
106152025-08-26T21:56:02.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
106162025-08-26T21:56:02.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
106172025-08-26T21:56:02.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
106182025-08-26T21:56:02.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
106192025-08-26T21:56:02.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
106202025-08-26T21:56:02.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
106212025-08-26T21:56:02.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
106222025-08-26T21:56:02.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
106232025-08-26T21:56:02.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
106242025-08-26T21:56:02.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
106252025-08-26T21:56:02.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
106262025-08-26T21:56:02.869ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
106272025-08-26T21:56:02.870ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
106282025-08-26T21:56:02.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
106292025-08-26T21:56:02.871ZINFOnexus (BackgroundTasks): instance reincarnation completed successfully background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:113 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0
106302025-08-26T21:56:02.871ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 11.448208ms iteration = 11 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
106312025-08-26T21:56:02.871ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
106322025-08-26T21:56:02.872ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 61 reason = Timeout
106332025-08-26T21:56:02.872ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
106342025-08-26T21:56:02.873ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
106352025-08-26T21:56:02.873ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
106362025-08-26T21:56:02.873ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 4.264176ms iteration = 21 status = {"errors":[],"garbage_collect_requested":[]}
106372025-08-26T21:56:02.873ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Fleet result = Ok(())
106382025-08-26T21:56:02.873ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
106392025-08-26T21:56:02.873ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
106402025-08-26T21:56:02.874ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
106412025-08-26T21:56:02.874ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
106422025-08-26T21:56:02.874ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
106432025-08-26T21:56:02.874ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
106442025-08-26T21:56:02.875ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
106452025-08-26T21:56:02.875ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
106462025-08-26T21:56:02.875ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
106472025-08-26T21:56:02.875ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
106482025-08-26T21:56:02.875ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
106492025-08-26T21:56:02.877ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
106502025-08-26T21:56:02.877ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
106512025-08-26T21:56:02.877ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
106522025-08-26T21:56:02.877ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
106532025-08-26T21:56:02.877ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
106542025-08-26T21:56:02.877ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
106552025-08-26T21:56:02.877ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 9.126186ms iteration = 21 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
106562025-08-26T21:56:02.878ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
106572025-08-26T21:56:02.878ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
106582025-08-26T21:56:02.878ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
106592025-08-26T21:56:02.879ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
106602025-08-26T21:56:02.879ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
106612025-08-26T21:56:02.880ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
106622025-08-26T21:56:02.880ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
106632025-08-26T21:56:02.880ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
106642025-08-26T21:56:02.880ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
106652025-08-26T21:56:02.880ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
106662025-08-26T21:56:02.880ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
106672025-08-26T21:56:02.881ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
106682025-08-26T21:56:02.881ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
106692025-08-26T21:56:02.881ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
106702025-08-26T21:56:02.881ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
106712025-08-26T21:56:02.881ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
106722025-08-26T21:56:02.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
106732025-08-26T21:56:02.882ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
106742025-08-26T21:56:02.882ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
106752025-08-26T21:56:02.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
106762025-08-26T21:56:02.883ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
106772025-08-26T21:56:02.883ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
106782025-08-26T21:56:02.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
106792025-08-26T21:56:02.896ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
106802025-08-26T21:56:02.897ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1466, tv_nsec: 755701060 } })
106812025-08-26T21:56:02.897ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
106822025-08-26T21:56:02.897ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1205 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
106832025-08-26T21:56:02.897ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
106842025-08-26T21:56:02.898ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:550 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
106852025-08-26T21:56:02.898ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 29.037135ms iteration = 21 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
106862025-08-26T21:56:02.899ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
106872025-08-26T21:56:02.899ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 30.205431ms iteration = 121 status = {"chicken_switches_updated":false}
106882025-08-26T21:56:02.899ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
106892025-08-26T21:56:02.900ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
106902025-08-26T21:56:02.902ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
106912025-08-26T21:56:02.902ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
106922025-08-26T21:56:02.902ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
106932025-08-26T21:56:02.902ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
106942025-08-26T21:56:02.903ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
106952025-08-26T21:56:02.903ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
106962025-08-26T21:56:02.904ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
106972025-08-26T21:56:02.904ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
106982025-08-26T21:56:02.905ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
106992025-08-26T21:56:02.906ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
107002025-08-26T21:56:02.906ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
107012025-08-26T21:56:02.907ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
107022025-08-26T21:56:02.907ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
107032025-08-26T21:56:02.907ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
107042025-08-26T21:56:02.907ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
107052025-08-26T21:56:02.907ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
107062025-08-26T21:56:02.907ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
107072025-08-26T21:56:02.908ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
107082025-08-26T21:56:02.908ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
107092025-08-26T21:56:02.908ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
107102025-08-26T21:56:02.908ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
107112025-08-26T21:56:02.909ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
107122025-08-26T21:56:02.910ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
107132025-08-26T21:56:02.911ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
107142025-08-26T21:56:02.911ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
107152025-08-26T21:56:02.911ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
107162025-08-26T21:56:02.911ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
107172025-08-26T21:56:02.911ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
107182025-08-26T21:56:02.911ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
107192025-08-26T21:56:02.911ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
107202025-08-26T21:56:02.912ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
107212025-08-26T21:56:02.913ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
107222025-08-26T21:56:02.914ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
107232025-08-26T21:56:02.914ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
107242025-08-26T21:56:02.914ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
107252025-08-26T21:56:02.915ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
107262025-08-26T21:56:02.915ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
107272025-08-26T21:56:02.916ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
107282025-08-26T21:56:02.917ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
107292025-08-26T21:56:02.918ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
107302025-08-26T21:56:02.918ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
107312025-08-26T21:56:02.918ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
107322025-08-26T21:56:02.918ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
107332025-08-26T21:56:02.919ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
107342025-08-26T21:56:02.919ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
107352025-08-26T21:56:02.919ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
107362025-08-26T21:56:02.920ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
107372025-08-26T21:56:02.920ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
107382025-08-26T21:56:02.924ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
107392025-08-26T21:56:02.925ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
107402025-08-26T21:56:02.934ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
107412025-08-26T21:56:02.934ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 8e720f02-dde8-4423-a6e3-91698373805a dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
107422025-08-26T21:56:02.934ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
107432025-08-26T21:56:02.935ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
107442025-08-26T21:56:02.936ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 67.659645ms iteration = 61 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
107452025-08-26T21:56:02.938ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
107462025-08-26T21:56:02.938ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
107472025-08-26T21:56:02.938ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
107482025-08-26T21:56:02.940ZINFOnexus (SagaRecoverer): listed in-progress sagas count = 0 file = nexus/src/app/background/tasks/saga_recovery.rs:473
107492025-08-26T21:56:02.940ZINFOnexus (SagaQuiesceHandle): saga recovery pass finished file = nexus/types/src/quiesce.rs:332 generation = 1
107502025-08-26T21:56:02.940ZDEBGnexus (BackgroundTasks): activation complete background_task = saga_recovery elapsed = 79.739847ms iteration = 2 status = {"last_pass":{"Success":{"nfailed":0,"nfound":0,"nrecovered":0,"nremoved":0,"nskipped":0}},"ntotal_failures":0,"ntotal_finished":0,"ntotal_recovered":0,"ntotal_sec_errors_bad_state":0,"ntotal_sec_errors_missing":0,"ntotal_started":0,"recent_failures":[],"recent_recoveries":[]}
107512025-08-26T21:56:02.940ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
107522025-08-26T21:56:02.941ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
107532025-08-26T21:56:02.941ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
107542025-08-26T21:56:02.941ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:138 found = 0 not_ready_to_be_deleted = 0
107552025-08-26T21:56:02.941ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 81.110308ms iteration = 11 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
107562025-08-26T21:56:02.941ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
107572025-08-26T21:56:02.941ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
107582025-08-26T21:56:02.941ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
107592025-08-26T21:56:02.942ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
107602025-08-26T21:56:02.942ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
107612025-08-26T21:56:02.942ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
107622025-08-26T21:56:02.942ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 70.107281ms iteration = 61 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
107632025-08-26T21:56:02.942ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
107642025-08-26T21:56:02.943ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
107652025-08-26T21:56:02.944ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
107662025-08-26T21:56:02.944ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
107672025-08-26T21:56:02.946ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 77.382404ms iteration = 21 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
107682025-08-26T21:56:02.946ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
107692025-08-26T21:56:02.946ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
107702025-08-26T21:56:02.946ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
107712025-08-26T21:56:02.946ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
107722025-08-26T21:56:02.946ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
107732025-08-26T21:56:02.946ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
107742025-08-26T21:56:02.946ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
107752025-08-26T21:56:02.946ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
107762025-08-26T21:56:02.946ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
107772025-08-26T21:56:02.947ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
107782025-08-26T21:56:02.947ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 78.98198ms iteration = 21 status = {"errors":[],"finish_invoked_ok":[]}
107792025-08-26T21:56:02.947ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
107802025-08-26T21:56:02.948ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
107812025-08-26T21:56:02.948ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
107822025-08-26T21:56:02.949ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 8e720f02-dde8-4423-a6e3-91698373805a dns_zone_name = control-plane.oxide.internal found_so_far = 59 version = 1
107832025-08-26T21:56:02.949ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
107842025-08-26T21:56:02.949ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2025-08-26 21:46:04.065217 UTC dns_group = internal generation = 1
107852025-08-26T21:56:02.949ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 116.761112ms iteration = 11 status = {"generation":1}
107862025-08-26T21:56:02.952ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
107872025-08-26T21:56:02.952ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
107882025-08-26T21:56:02.952ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
107892025-08-26T21:56:02.954ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
107902025-08-26T21:56:02.954ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
107912025-08-26T21:56:02.957ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
107922025-08-26T21:56:02.957ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
107932025-08-26T21:56:02.958ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 89.768843ms iteration = 21 status = {"errors":[],"requests_created_ok":[]}
107942025-08-26T21:56:02.961ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
107952025-08-26T21:56:02.961ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
107962025-08-26T21:56:02.964ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
107972025-08-26T21:56:02.965ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
107982025-08-26T21:56:02.967ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
107992025-08-26T21:56:02.967ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
108002025-08-26T21:56:02.968ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
108012025-08-26T21:56:02.968ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
108022025-08-26T21:56:02.969ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
108032025-08-26T21:56:02.969ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
108042025-08-26T21:56:02.971ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
108052025-08-26T21:56:02.971ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
108062025-08-26T21:56:02.975ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
108072025-08-26T21:56:02.975ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
108082025-08-26T21:56:02.975ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
108092025-08-26T21:56:02.975ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
108102025-08-26T21:56:02.975ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = dfb404ab-acb8-43de-afe8-b429af0c161d dns_zone_name = oxide.test found_so_far = 0 version = 2
108112025-08-26T21:56:02.975ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
108122025-08-26T21:56:02.976ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
108132025-08-26T21:56:02.977ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:407
108142025-08-26T21:56:02.977ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 108.758652ms iteration = 21 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
108152025-08-26T21:56:02.977ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
108162025-08-26T21:56:02.979ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
108172025-08-26T21:56:02.979ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
108182025-08-26T21:56:02.979ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
108192025-08-26T21:56:02.981ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
108202025-08-26T21:56:02.982ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
108212025-08-26T21:56:02.982ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
108222025-08-26T21:56:02.983ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
108232025-08-26T21:56:02.983ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
108242025-08-26T21:56:02.983ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
108252025-08-26T21:56:02.983ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
108262025-08-26T21:56:02.984ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
108272025-08-26T21:56:02.984ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
108282025-08-26T21:56:02.984ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
108292025-08-26T21:56:02.985ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = dfb404ab-acb8-43de-afe8-b429af0c161d dns_zone_name = oxide.test found_so_far = 4 version = 2
108302025-08-26T21:56:02.985ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
108312025-08-26T21:56:02.985ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2025-08-26 21:46:05.296205 UTC dns_group = external generation = 2
108322025-08-26T21:56:02.985ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 153.161163ms iteration = 11 status = {"generation":2}
108332025-08-26T21:56:02.986ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
108342025-08-26T21:56:02.988ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
108352025-08-26T21:56:02.988ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
108362025-08-26T21:56:02.988ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
108372025-08-26T21:56:02.991ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
108382025-08-26T21:56:02.991ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
108392025-08-26T21:56:02.991ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1213 zone_address = fd00:1122:3344:101::2
108402025-08-26T21:56:02.991ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
108412025-08-26T21:56:02.992ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
108422025-08-26T21:56:02.992ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "8cb796d1-6154-4d6d-897c-6b2316d37e74", "content-length": "26", "date": "Tue, 26 Aug 2025 21:56:02 GMT"} })
108432025-08-26T21:56:02.992ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1216 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
108442025-08-26T21:56:02.994ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
108452025-08-26T21:56:02.997ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
108462025-08-26T21:56:02.997ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
108472025-08-26T21:56:03.000ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
108482025-08-26T21:56:03.000ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
108492025-08-26T21:56:03.002ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
108502025-08-26T21:56:03.002ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
108512025-08-26T21:56:03.002ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
108522025-08-26T21:56:03.002ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
108532025-08-26T21:56:03.004ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 135.366012ms iteration = 21 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
108542025-08-26T21:56:03.004ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-08-26T21:45:59.866217Z, time_modified: 2025-08-26T21:45:59.866217Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 9396aa55-de6f-4d6f-96db-d28e1c76fa9c, db_silo: Silo { identity: SiloIdentity { id: 9396aa55-de6f-4d6f-96db-d28e1c76fa9c, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-08-26T21:46:05.245521Z, time_modified: 2025-08-26T21:46:05.245521Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "e8529d720e4b0de099753dcdb373abb870af04baacd7c7dd850f5254858081af" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 9396aa55-de6f-4d6f-96db-d28e1c76fa9c, db_silo: Silo { identity: SiloIdentity { id: 9396aa55-de6f-4d6f-96db-d28e1c76fa9c, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-08-26T21:46:05.245521Z, time_modified: 2025-08-26T21:46:05.245521Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "e8529d720e4b0de099753dcdb373abb870af04baacd7c7dd850f5254858081af" }] }) }
108552025-08-26T21:56:03.004ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 171.70416ms iteration = 11 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"9396aa55-de6f-4d6f-96db-d28e1c76fa9c","tls_certs":["e8529d720e4b0de099753dcdb373abb870af04baacd7c7dd850f5254858081af"]}},"default_endpoint":{"silo_id":"9396aa55-de6f-4d6f-96db-d28e1c76fa9c","tls_certs":["e8529d720e4b0de099753dcdb373abb870af04baacd7c7dd850f5254858081af"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
108562025-08-26T21:56:03.005ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
108572025-08-26T21:56:03.006ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
108582025-08-26T21:56:03.006ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
108592025-08-26T21:56:03.006ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
108602025-08-26T21:56:03.006ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
108612025-08-26T21:56:03.006ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
108622025-08-26T21:56:03.008ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
108632025-08-26T21:56:03.009ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
108642025-08-26T21:56:03.010ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
108652025-08-26T21:56:03.011ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
108662025-08-26T21:56:03.013ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
108672025-08-26T21:56:03.013ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
108682025-08-26T21:56:03.015ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
108692025-08-26T21:56:03.020ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1213 zone_address = fd00:1122:3344:101::2
108702025-08-26T21:56:03.020ZDEBGnexus (MgsClient): client request background_task = nat_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
108712025-08-26T21:56:03.020ZDEBGnexus (MgsClient): client response background_task = nat_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "905fa044-9c75-42fe-9b0e-256ed0e3b141", "content-length": "26", "date": "Tue, 26 Aug 2025 21:56:02 GMT"} })
108722025-08-26T21:56:03.020ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1216 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
108732025-08-26T21:56:03.020ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
108742025-08-26T21:56:03.020ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
108752025-08-26T21:56:03.021ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
108762025-08-26T21:56:03.022ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
108772025-08-26T21:56:03.023ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
108782025-08-26T21:56:03.023ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
108792025-08-26T21:56:03.025ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
108802025-08-26T21:56:03.025ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
108812025-08-26T21:56:03.027ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
108822025-08-26T21:56:03.032ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
108832025-08-26T21:56:03.032ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
108842025-08-26T21:56:03.032ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
108852025-08-26T21:56:03.041ZDEBGnexus (service-firewall-rule-progator): resolved names for firewall rules background_task = service_firewall_rule_propagation instance_interfaces = {} subnet_interfaces = {Name("external-dns"): [NetworkInterface { id: 0106180c-e6f8-4d65-969e-4fbbddf596a1, kind: Service { id: 60abf731-31e1-4111-a3e2-65da89eaed87 }, name: Name("external-dns-60abf731-31e1-4111-a3e2-65da89eaed87"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 156, 81])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: b69ae953-8f67-4cb3-9521-db9e707a6f2d, kind: Service { id: 84c849eb-0949-41d9-ad02-9a5509144564 }, name: Name("external-dns-84c849eb-0949-41d9-ad02-9a5509144564"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 240, 136])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], Name("nexus"): [NetworkInterface { id: 0802405a-a5f2-402c-94da-ccafb58bccb5, kind: Service { id: 4996dcc4-c530-4f64-b407-fa372462bd9e }, name: Name("nexus-4996dcc4-c530-4f64-b407-fa372462bd9e"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 186, 56])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 1acb6bfd-558b-4271-bbf5-58e0d089b710, kind: Service { id: 0205eaf0-3fc6-4ad5-b09b-732af233e9c1 }, name: Name("nexus-0205eaf0-3fc6-4ad5-b09b-732af233e9c1"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 243, 142])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: cbb99fa2-edc6-497d-9c47-8dd2642f8945, kind: Service { id: 5d0959e5-47a6-4bf3-a52b-57950b9d900d }, name: Name("nexus-5d0959e5-47a6-4bf3-a52b-57950b9d900d"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 162, 80])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }]} subnet_networks = {Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })], Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })]} vpc_interfaces = {}
108862025-08-26T21:56:03.041ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
108872025-08-26T21:56:03.041ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
108882025-08-26T21:56:03.043ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
108892025-08-26T21:56:03.047ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Fleet result = Ok(())
108902025-08-26T21:56:03.047ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
108912025-08-26T21:56:03.047ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
108922025-08-26T21:56:03.049ZDEBGnexus (service-firewall-rule-progator): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified. background_task = service_firewall_rule_propagation
108932025-08-26T21:56:03.049ZDEBGnexus (service-firewall-rule-progator): resolved firewall rules for sled agents background_task = service_firewall_rule_propagation sled_agent_rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 0106180c-e6f8-4d65-969e-4fbbddf596a1, kind: Service { id: 60abf731-31e1-4111-a3e2-65da89eaed87 }, name: Name("external-dns-60abf731-31e1-4111-a3e2-65da89eaed87"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 156, 81])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: b69ae953-8f67-4cb3-9521-db9e707a6f2d, kind: Service { id: 84c849eb-0949-41d9-ad02-9a5509144564 }, name: Name("external-dns-84c849eb-0949-41d9-ad02-9a5509144564"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 240, 136])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 0802405a-a5f2-402c-94da-ccafb58bccb5, kind: Service { id: 4996dcc4-c530-4f64-b407-fa372462bd9e }, name: Name("nexus-4996dcc4-c530-4f64-b407-fa372462bd9e"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 186, 56])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 1acb6bfd-558b-4271-bbf5-58e0d089b710, kind: Service { id: 0205eaf0-3fc6-4ad5-b09b-732af233e9c1 }, name: Name("nexus-0205eaf0-3fc6-4ad5-b09b-732af233e9c1"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 243, 142])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: cbb99fa2-edc6-497d-9c47-8dd2642f8945, kind: Service { id: 5d0959e5-47a6-4bf3-a52b-57950b9d900d }, name: Name("nexus-5d0959e5-47a6-4bf3-a52b-57950b9d900d"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 162, 80])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 0106180c-e6f8-4d65-969e-4fbbddf596a1, kind: Service { id: 60abf731-31e1-4111-a3e2-65da89eaed87 }, name: Name("external-dns-60abf731-31e1-4111-a3e2-65da89eaed87"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 156, 81])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: b69ae953-8f67-4cb3-9521-db9e707a6f2d, kind: Service { id: 84c849eb-0949-41d9-ad02-9a5509144564 }, name: Name("external-dns-84c849eb-0949-41d9-ad02-9a5509144564"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 240, 136])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: None, filter_protocols: Some([Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 3, code: Some(IcmpParamRange { first: 3, last: 4 }) })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 5, code: None })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 11, code: None }))]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 0802405a-a5f2-402c-94da-ccafb58bccb5, kind: Service { id: 4996dcc4-c530-4f64-b407-fa372462bd9e }, name: Name("nexus-4996dcc4-c530-4f64-b407-fa372462bd9e"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 186, 56])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 1acb6bfd-558b-4271-bbf5-58e0d089b710, kind: Service { id: 0205eaf0-3fc6-4ad5-b09b-732af233e9c1 }, name: Name("nexus-0205eaf0-3fc6-4ad5-b09b-732af233e9c1"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 243, 142])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: cbb99fa2-edc6-497d-9c47-8dd2642f8945, kind: Service { id: 5d0959e5-47a6-4bf3-a52b-57950b9d900d }, name: Name("nexus-5d0959e5-47a6-4bf3-a52b-57950b9d900d"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 162, 80])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }]
108942025-08-26T21:56:03.049ZDEBGnexus (service-firewall-rule-progator): resolved 3 rules for sleds background_task = service_firewall_rule_propagation
108952025-08-26T21:56:03.058ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
108962025-08-26T21:56:03.058ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
108972025-08-26T21:56:03.058ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 189.297657ms iteration = 21 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
108982025-08-26T21:56:03.080ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
108992025-08-26T21:56:03.080ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
109002025-08-26T21:56:03.081ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
109012025-08-26T21:56:03.081ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
109022025-08-26T21:56:03.090ZINFOnexus (BackgroundTasks): removed inventory collection background_task = inventory_collection collection_id = 98fcdfec-5acc-4b21-be95-18fc820aa3d1 file = nexus/db-queries/src/db/datastore/inventory.rs:2266 nboot_partitions = 0 ncabooses = 12 nclickhouse_keeper_membership = 0 ncockroach_status = 5 ncollections = 1 nconfig_reconcilers = 1 ndatasets = 84 nerrors = 0 nhost_phase1_active_slots = 1 nhost_phase1_flash_hashes = 2 ninternal_dns = 3 nlast_reconciliation_dataset_results = 66 nlast_reconciliation_disk_results = 9 nlast_reconciliation_orphaned_datasets = 0 nlast_reconciliation_zone_results = 28 nmupdate_override_non_boot = 0 nntp_timesync = 1 nnvme_disk_firmware = 11 nomicron_sled_config_datasets = 66 nomicron_sled_config_disks = 9 nomicron_sled_config_zone_nics = 6 nomicron_sled_config_zones = 28 nomicron_sled_configs = 1 nphysical_disks = 11 nrot_pages = 8 nrots = 2 nsled_agents = 1 nsps = 2 nzone_manifest_non_boot = 0 nzone_manifest_zones = 0 nzpools = 9
109032025-08-26T21:56:03.090ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
109042025-08-26T21:56:03.091ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
109052025-08-26T21:56:03.106ZDEBGnexus (BackgroundTasks): inventory_prune_one: eligible for removal background_task = inventory_collection candidates = [(f37c741e-e5eb-4be5-8e12-bbe92da18b97, 0), (22ce40dd-e0b9-460a-8151-60c0f8e6fa4f, 0), (08443897-9b9f-4f71-92e4-66f0b6ae2a76, 0), (28fbc916-7029-4e55-95e8-37ed39eb5e3f, 0)] collection_id = f37c741e-e5eb-4be5-8e12-bbe92da18b97
109062025-08-26T21:56:03.106ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
109072025-08-26T21:56:03.106ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
109082025-08-26T21:56:03.115ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
109092025-08-26T21:56:03.117ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 248.523332ms iteration = 22 status = {"physical_disks_added":0}
109102025-08-26T21:56:03.123ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1213 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 zone_address = fd00:1122:3344:101::2
109112025-08-26T21:56:03.123ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
109122025-08-26T21:56:03.124ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "ba331962-8335-4ec0-938b-27ae0844a34c", "content-length": "26", "date": "Tue, 26 Aug 2025 21:56:03 GMT"} })
109132025-08-26T21:56:03.124ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1216 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
109142025-08-26T21:56:03.127ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
109152025-08-26T21:56:03.135ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
109162025-08-26T21:56:03.146ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
109172025-08-26T21:56:03.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
109182025-08-26T21:56:03.149ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
109192025-08-26T21:56:03.151ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
109202025-08-26T21:56:03.151ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
109212025-08-26T21:56:03.151ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
109222025-08-26T21:56:03.153ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
109232025-08-26T21:56:03.153ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
109242025-08-26T21:56:03.154ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
109252025-08-26T21:56:03.155ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
109262025-08-26T21:56:03.155ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
109272025-08-26T21:56:03.157ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
109282025-08-26T21:56:03.157ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
109292025-08-26T21:56:03.157ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
109302025-08-26T21:56:03.157ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
109312025-08-26T21:56:03.160ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
109322025-08-26T21:56:03.161ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
109332025-08-26T21:56:03.161ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: dd0c655a-95c2-48fc-b4c3-9733ed4b0faa, lookup_type: ById(dd0c655a-95c2-48fc-b4c3-9733ed4b0faa) } result = Ok(())
109342025-08-26T21:56:03.161ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
109352025-08-26T21:56:03.161ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
109362025-08-26T21:56:03.163ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
109372025-08-26T21:56:03.164ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 295.529771ms iteration = 21 status = 0
109382025-08-26T21:56:03.166ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
109392025-08-26T21:56:03.167ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
109402025-08-26T21:56:03.167ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
109412025-08-26T21:56:03.168ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
109422025-08-26T21:56:03.171ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
109432025-08-26T21:56:03.171ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
109442025-08-26T21:56:03.171ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
109452025-08-26T21:56:03.176ZINFOnexus (BackgroundTasks): ensured all Crucible datasets present in inventory have database records background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/crucible_dataset.rs:113 num_already_exist = 9 num_inserted = 0 num_not_in_inventory = 0
109462025-08-26T21:56:03.176ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_rendezvous elapsed = 316.094176ms iteration = 4 status = {"blueprint_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","inventory_collection_id":"28fbc916-7029-4e55-95e8-37ed39eb5e3f","stats":{"crucible_dataset":{"num_already_exist":9,"num_inserted":0,"num_not_in_inventory":0},"debug_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0}}}
109472025-08-26T21:56:03.191ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
109482025-08-26T21:56:03.192ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "172e29e9-fe73-4b0d-adc3-5f3121b1a5d6", "content-length": "691", "date": "Tue, 26 Aug 2025 21:56:03 GMT"} })
109492025-08-26T21:56:03.192ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
109502025-08-26T21:56:03.192ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
109512025-08-26T21:56:03.192ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
109522025-08-26T21:56:03.193ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 325.044427ms iteration = 21 status = {}
109532025-08-26T21:56:03.221ZINFOnexus (BackgroundTasks): removed inventory collection background_task = inventory_collection collection_id = f37c741e-e5eb-4be5-8e12-bbe92da18b97 file = nexus/db-queries/src/db/datastore/inventory.rs:2266 nboot_partitions = 0 ncabooses = 12 nclickhouse_keeper_membership = 0 ncockroach_status = 5 ncollections = 1 nconfig_reconcilers = 1 ndatasets = 84 nerrors = 0 nhost_phase1_active_slots = 1 nhost_phase1_flash_hashes = 2 ninternal_dns = 3 nlast_reconciliation_dataset_results = 66 nlast_reconciliation_disk_results = 9 nlast_reconciliation_orphaned_datasets = 0 nlast_reconciliation_zone_results = 28 nmupdate_override_non_boot = 0 nntp_timesync = 1 nnvme_disk_firmware = 11 nomicron_sled_config_datasets = 66 nomicron_sled_config_disks = 9 nomicron_sled_config_zone_nics = 6 nomicron_sled_config_zones = 28 nomicron_sled_configs = 1 nphysical_disks = 11 nrot_pages = 8 nrots = 2 nsled_agents = 1 nsps = 2 nzone_manifest_non_boot = 0 nzone_manifest_zones = 0 nzpools = 9
109542025-08-26T21:56:03.221ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
109552025-08-26T21:56:03.221ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
109562025-08-26T21:56:03.225ZDEBGnexus (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(22ce40dd-e0b9-460a-8151-60c0f8e6fa4f, 0), (08443897-9b9f-4f71-92e4-66f0b6ae2a76, 0), (28fbc916-7029-4e55-95e8-37ed39eb5e3f, 0)]
109572025-08-26T21:56:03.226ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1467, tv_nsec: 84914662 } })
109582025-08-26T21:56:03.232ZDEBGnexus (service-firewall-rule-progator): resolved sleds for vpc oxide-services background_task = service_firewall_rule_propagation vpc_to_sled = [Sled { identity: SledIdentity { id: dd0c655a-95c2-48fc-b4c3-9733ed4b0faa, time_created: 2025-08-26T21:46:02.113579Z, time_modified: 2025-08-26T21:46:02.113579Z }, time_deleted: None, rcgen: Generation(Generation(19)), rack_id: e37a46f2-f789-4ca4-9841-f8c6dd705964, is_scrimlet: true, serial_number: "sock", part_number: "i86pc", revision: SqlU32(0), usable_hardware_threads: SqlU32(48), usable_physical_ram: ByteCount(ByteCount(102952026112)), reservoir_size: ByteCount(ByteCount(56094621696)), ip: fd00:1122:3344:101::1, port: SqlU16(12345), last_used_address: fd00:1122:3344:101::ffff, policy: InService, state: Active, sled_agent_gen: Generation(Generation(1)), repo_depot_port: SqlU16(12348), cpu_family: AmdMilan }]
109592025-08-26T21:56:03.232ZDEBGnexus (service-firewall-rule-progator): sending firewall rules to sled agents background_task = service_firewall_rule_propagation
109602025-08-26T21:56:03.272ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
109612025-08-26T21:56:03.272ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
109622025-08-26T21:56:03.274ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
109632025-08-26T21:56:03.276ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Fleet result = Ok(())
109642025-08-26T21:56:03.276ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
109652025-08-26T21:56:03.276ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
109662025-08-26T21:56:03.278ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
109672025-08-26T21:56:03.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
109682025-08-26T21:56:03.280ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
109692025-08-26T21:56:03.282ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Fleet result = Ok(())
109702025-08-26T21:56:03.282ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
109712025-08-26T21:56:03.282ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
109722025-08-26T21:56:03.284ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = tuf_artifact_replication body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/artifacts-config
109732025-08-26T21:56:03.285ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = tuf_artifact_replication result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/artifacts-config", status: 204, headers: {"x-request-id": "b027fe3a-1540-4b89-86ea-95d819ddda5f", "date": "Tue, 26 Aug 2025 21:56:03 GMT"} })
109742025-08-26T21:56:03.285ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = tuf_artifact_replication body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/artifacts
109752025-08-26T21:56:03.285ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = tuf_artifact_replication result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/artifacts", status: 200, headers: {"content-type": "application/json", "x-request-id": "e8408b0b-5e90-4ddc-bac6-aed39135e9be", "content-length": "26", "date": "Tue, 26 Aug 2025 21:56:03 GMT"} })
109762025-08-26T21:56:03.285ZINFOnexus (BackgroundTasks): Successfully got artifact list background_task = tuf_artifact_replication file = nexus/src/app/background/tasks/tuf_artifact_replication.rs:672 sled = http://[fd00:1122:3344:101::1]:12345
109772025-08-26T21:56:03.288ZDEBGnexus (BackgroundTasks): activation complete background_task = tuf_artifact_replication elapsed = 427.442695ms iteration = 3 status = {"generation":1,"last_run_counters":{"copy_err":0,"copy_ok":0,"list_err":0,"list_ok":1,"put_config_err":0,"put_config_ok":1,"put_err":0,"put_ok":0},"lifetime_counters":{"copy_err":0,"copy_ok":0,"list_err":0,"list_ok":3,"put_config_err":0,"put_config_ok":3,"put_err":0,"put_ok":0},"local_repos":0,"request_debug_ringbuf":[{"generation":1,"operation":"put_config","target_sled":"dd0c655a-95c2-48fc-b4c3-9733ed4b0faa","time":"2025-08-26T21:46:03.195562885Z"},{"operation":"list","target_sled":"dd0c655a-95c2-48fc-b4c3-9733ed4b0faa","time":"2025-08-26T21:46:03.196425124Z"},{"generation":1,"operation":"put_config","target_sled":"dd0c655a-95c2-48fc-b4c3-9733ed4b0faa","time":"2025-08-26T21:51:03.181001208Z"},{"operation":"list","target_sled":"dd0c655a-95c2-48fc-b4c3-9733ed4b0faa","time":"2025-08-26T21:51:03.181295604Z"},{"generation":1,"operation":"put_config","target_sled":"dd0c655a-95c2-48fc-b4c3-9733ed4b0faa","time":"2025-08-26T21:56:03.285672355Z"},{"operation":"list","target_sled":"dd0c655a-95c2-48fc-b4c3-9733ed4b0faa","time":"2025-08-26T21:56:03.285915990Z"}]}
109782025-08-26T21:56:03.294ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
109792025-08-26T21:56:03.295ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
109802025-08-26T21:56:03.297ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 464.415009ms iteration = 21 status = null
109812025-08-26T21:56:03.428ZINFOnexus (BackgroundTasks): VPC route manager sled dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
109822025-08-26T21:56:03.428ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
109832025-08-26T21:56:03.428ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "60fe4490-0074-4d20-ae65-4758b7167e24", "content-length": "372", "date": "Tue, 26 Aug 2025 21:56:03 GMT"} })
109842025-08-26T21:56:03.428ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
109852025-08-26T21:56:03.429ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
109862025-08-26T21:56:03.432ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 599.932753ms iteration = 21 status = {}
109872025-08-26T21:56:03.458ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
109882025-08-26T21:56:03.478ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _cockroach._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_cockroach._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("5362db44-dfc6-41c1-8ec8-32ab3e86b7b3.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("5e2b44ef-e7e6-47ec-9087-a2fb44127ede.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("70604151-9a25-4539-bcae-b8d33d91f6a5.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("be83bf79-a7c5-40a7-be80-31ec2f206604.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("d35c8171-47d8-4704-93d7-f07739a9281c.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("d35c8171-47d8-4704-93d7-f07739a9281c.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::7)) }], valid_until: Instant { tv_sec: 1467, tv_nsec: 337365972 } })
109892025-08-26T21:56:03.479ZDEBGnexus (BackgroundTasks): begin collection background_task = inventory_collection
109902025-08-26T21:56:03.479ZDEBGnexus (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[fd00:1122:3344:101::2]:12225
109912025-08-26T21:56:03.479ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/ignition
109922025-08-26T21:56:03.479ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "8a1958c5-0ed0-4765-b259-a23d375bfa8a", "content-length": "476", "date": "Tue, 26 Aug 2025 21:56:02 GMT"} })
109932025-08-26T21:56:03.479ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0
109942025-08-26T21:56:03.480ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "29fe2901-65e1-45dd-92b2-80b3015662f9", "content-length": "734", "date": "Tue, 26 Aug 2025 21:56:03 GMT"} })
109952025-08-26T21:56:03.480ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0
109962025-08-26T21:56:03.480ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "77f0a66f-fd1a-4670-938c-633b4e2d529d", "content-length": "111", "date": "Tue, 26 Aug 2025 21:56:02 GMT"} })
109972025-08-26T21:56:03.480ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1
109982025-08-26T21:56:03.481ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "99860477-9011-4f41-b451-e99d100af3fd", "content-length": "111", "date": "Tue, 26 Aug 2025 21:56:02 GMT"} })
109992025-08-26T21:56:03.481ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0
110002025-08-26T21:56:03.481ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "abcaf8c6-cc19-4441-856f-0d0f4304b8c8", "content-length": "163", "date": "Tue, 26 Aug 2025 21:56:03 GMT"} })
110012025-08-26T21:56:03.481ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1
110022025-08-26T21:56:03.481ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "a4fc068f-5d1e-484d-bbf5-2dd6fa47c89f", "content-length": "163", "date": "Tue, 26 Aug 2025 21:56:03 GMT"} })
110032025-08-26T21:56:03.481ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0
110042025-08-26T21:56:03.482ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "334c8ee2-d95c-41b0-aa78-77fc0aab5231", "content-length": "166", "date": "Tue, 26 Aug 2025 21:56:03 GMT"} })
110052025-08-26T21:56:03.482ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1
110062025-08-26T21:56:03.482ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c76e7d44-aae3-483e-a4f3-9ad9a58b8a08", "content-length": "166", "date": "Tue, 26 Aug 2025 21:56:03 GMT"} })
110072025-08-26T21:56:03.482ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa
110082025-08-26T21:56:03.482ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a927a47a-a032-43d0-b72f-04c5b931f2c0", "content-length": "702", "date": "Tue, 26 Aug 2025 21:56:03 GMT"} })
110092025-08-26T21:56:03.482ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
110102025-08-26T21:56:03.483ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
110112025-08-26T21:56:03.483ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "985bfa01-59fa-42bd-a348-2aa328ff469e", "content-length": "727", "date": "Tue, 26 Aug 2025 21:56:03 GMT"} })
110122025-08-26T21:56:03.483ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
110132025-08-26T21:56:03.483ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
110142025-08-26T21:56:03.483ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "5125779a-73c2-46e7-b296-499c9a7b8ceb", "content-length": "729", "date": "Tue, 26 Aug 2025 21:56:03 GMT"} })
110152025-08-26T21:56:03.483ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
110162025-08-26T21:56:03.483ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "c67a41b6-4c79-4276-89b6-1bfcf8e5cd1d", "date": "Tue, 26 Aug 2025 21:56:03 GMT"} })
110172025-08-26T21:56:03.483ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
110182025-08-26T21:56:03.483ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "cc1b78e2-688a-4dc3-a720-7dd33572040e", "content-length": "728", "date": "Tue, 26 Aug 2025 21:56:03 GMT"} })
110192025-08-26T21:56:03.483ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0
110202025-08-26T21:56:03.483ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "36254395-26ff-4a3e-9a0b-a83a99a11d45", "content-length": "722", "date": "Tue, 26 Aug 2025 21:56:03 GMT"} })
110212025-08-26T21:56:03.483ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/active-slot
110222025-08-26T21:56:03.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
110232025-08-26T21:56:03.484ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/active-slot", status: 200, headers: {"content-type": "application/json", "x-request-id": "de364854-7061-41a8-bb10-f11ee865f860", "content-length": "10", "date": "Tue, 26 Aug 2025 21:56:03 GMT"} })
110242025-08-26T21:56:03.484ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/0
110252025-08-26T21:56:03.484ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "5705fe11-f419-4391-b131-c7125617b622", "content-length": "144", "date": "Tue, 26 Aug 2025 21:56:03 GMT"} })
110262025-08-26T21:56:03.484ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/1
110272025-08-26T21:56:03.484ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c9a8fedf-64da-495f-b583-4bb21140b205", "content-length": "147", "date": "Tue, 26 Aug 2025 21:56:03 GMT"} })
110282025-08-26T21:56:03.484ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0
110292025-08-26T21:56:03.484ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "7efd626c-eb3e-40b4-abd6-f625f6a1adab", "content-length": "109", "date": "Tue, 26 Aug 2025 21:56:03 GMT"} })
110302025-08-26T21:56:03.484ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1
110312025-08-26T21:56:03.485ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "15fe9c46-d276-4e46-8e46-167697e64283", "content-length": "109", "date": "Tue, 26 Aug 2025 21:56:03 GMT"} })
110322025-08-26T21:56:03.485ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0
110332025-08-26T21:56:03.485ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "8bb9be11-a3b9-4e99-8ed4-20ec1a25dc78", "content-length": "163", "date": "Tue, 26 Aug 2025 21:56:03 GMT"} })
110342025-08-26T21:56:03.485ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1
110352025-08-26T21:56:03.485ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "17e8a07a-5f78-4896-a7be-661e250fb0ac", "content-length": "163", "date": "Tue, 26 Aug 2025 21:56:03 GMT"} })
110362025-08-26T21:56:03.485ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0
110372025-08-26T21:56:03.486ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "8dd94ea6-7007-4e39-960b-68c91e99f8c3", "content-length": "166", "date": "Tue, 26 Aug 2025 21:56:03 GMT"} })
110382025-08-26T21:56:03.486ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
110392025-08-26T21:56:03.486ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1
110402025-08-26T21:56:03.486ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
110412025-08-26T21:56:03.486ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "55dcbdc4-198a-4119-a13c-a2fb0c33c730", "content-length": "166", "date": "Tue, 26 Aug 2025 21:56:03 GMT"} })
110422025-08-26T21:56:03.486ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa
110432025-08-26T21:56:03.486ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "da8428e1-08df-40b6-8751-a7ed3a98def4", "content-length": "702", "date": "Tue, 26 Aug 2025 21:56:03 GMT"} })
110442025-08-26T21:56:03.486ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
110452025-08-26T21:56:03.487ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b44787f8-16f3-42d5-97a0-b4e4444b891e", "content-length": "727", "date": "Tue, 26 Aug 2025 21:56:03 GMT"} })
110462025-08-26T21:56:03.487ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
110472025-08-26T21:56:03.487ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "dc123d02-bd45-470a-acce-087fbd641a0f", "content-length": "729", "date": "Tue, 26 Aug 2025 21:56:03 GMT"} })
110482025-08-26T21:56:03.487ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
110492025-08-26T21:56:03.487ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "194bea8b-a6ee-4206-847c-31e1331ef26f", "content-length": "728", "date": "Tue, 26 Aug 2025 21:56:03 GMT"} })
110502025-08-26T21:56:03.487ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
110512025-08-26T21:56:03.487ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
110522025-08-26T21:56:03.489ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
110532025-08-26T21:56:03.489ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
110542025-08-26T21:56:03.489ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
110552025-08-26T21:56:03.491ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
110562025-08-26T21:56:03.491ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
110572025-08-26T21:56:03.491ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
110582025-08-26T21:56:03.492ZINFOnexus (BackgroundTasks): VPC route manager sled dd0c655a-95c2-48fc-b4c3-9733ed4b0faa push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
110592025-08-26T21:56:03.492ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
110602025-08-26T21:56:03.492ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
110612025-08-26T21:56:03.493ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "90ef9275-f0b4-4369-9532-d31d41330df0", "date": "Tue, 26 Aug 2025 21:56:03 GMT"} })
110622025-08-26T21:56:03.494ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 625.651813ms iteration = 21 status = {}
110632025-08-26T21:56:03.494ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
110642025-08-26T21:56:03.494ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
110652025-08-26T21:56:03.495ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
110662025-08-26T21:56:03.524ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
110672025-08-26T21:56:03.524ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "47fddb74-64f7-4b1b-b297-35f3fb1db320", "date": "Tue, 26 Aug 2025 21:56:03 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
110682025-08-26T21:56:03.524ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
110692025-08-26T21:56:03.526ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "38a1cc48-2bed-4714-a4d8-e346a72cad0e", "date": "Tue, 26 Aug 2025 21:56:03 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
110702025-08-26T21:56:03.526ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
110712025-08-26T21:56:03.527ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
110722025-08-26T21:56:03.527ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 695.16067ms iteration = 13 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
110732025-08-26T21:56:03.584ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
110742025-08-26T21:56:03.584ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
110752025-08-26T21:56:03.586ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
110762025-08-26T21:56:03.586ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
110772025-08-26T21:56:03.588ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
110782025-08-26T21:56:03.591ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Sled { parent: Fleet, key: dd0c655a-95c2-48fc-b4c3-9733ed4b0faa, lookup_type: ById(dd0c655a-95c2-48fc-b4c3-9733ed4b0faa) } result = Ok(())
110792025-08-26T21:56:03.591ZDEBGnexus (service-firewall-rule-progator): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = service_firewall_rule_propagation body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules
110802025-08-26T21:56:03.592ZDEBGnexus (service-firewall-rule-progator): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = service_firewall_rule_propagation result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules", status: 204, headers: {"x-request-id": "dcaf4592-8499-4c5b-a64f-4cb2db674eb7", "date": "Tue, 26 Aug 2025 21:56:03 GMT"} })
110812025-08-26T21:56:03.594ZINFOnexus (service-firewall-rule-progator): updated firewall rules on 1 sleds background_task = service_firewall_rule_propagation file = nexus/networking/src/firewall_rules.rs:491
110822025-08-26T21:56:03.594ZDEBGnexus (service-firewall-rule-progator): successfully propagated service firewall rules background_task = service_firewall_rule_propagation elapsed = 733.701769ms
110832025-08-26T21:56:03.594ZDEBGnexus (BackgroundTasks): activation complete background_task = service_firewall_rule_propagation elapsed = 733.708629ms iteration = 3 status = {}
110842025-08-26T21:56:03.606ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
110852025-08-26T21:56:03.606ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "0991d483-53dd-4ef5-907d-9f0bd9212186", "date": "Tue, 26 Aug 2025 21:56:03 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
110862025-08-26T21:56:03.606ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
110872025-08-26T21:56:03.608ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "fb4cb20e-a142-4a1c-9122-6e2846789376", "date": "Tue, 26 Aug 2025 21:56:03 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
110882025-08-26T21:56:03.608ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
110892025-08-26T21:56:03.610ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "25962782-c147-40de-9d49-84e12752a6c2", "date": "Tue, 26 Aug 2025 21:56:03 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
110902025-08-26T21:56:03.610ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
110912025-08-26T21:56:03.612ZDEBGnexus (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1]:12345
110922025-08-26T21:56:03.612ZDEBGnexus (BackgroundTasks): client request SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/inventory
110932025-08-26T21:56:03.612ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
110942025-08-26T21:56:03.612ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 779.859689ms iteration = 13 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
110952025-08-26T21:56:03.646ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
110962025-08-26T21:56:03.646ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
110972025-08-26T21:56:03.648ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: d24c5c55-3a21-4a91-aeb4-45362e822df6, rack_id: e37a46f2-f789-4ca4-9841-f8c6dd705964, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(88d84f37-a4c8-4abc-af1f-27987b0d39d9) } rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0
110982025-08-26T21:56:03.648ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
110992025-08-26T21:56:03.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
111002025-08-26T21:56:03.659ZDEBGnexus (BackgroundTasks): client response SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "5a31e3fc-6bcb-455a-8141-6a745ddf747f", "content-length": "88498", "date": "Tue, 26 Aug 2025 21:56:03 GMT"} })
111012025-08-26T21:56:03.661ZDEBGnexus (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
111022025-08-26T21:56:03.661ZDEBGnexus (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
111032025-08-26T21:56:03.661ZDEBGnexus (BackgroundTasks): begin collection from CockroachDB nodes background_task = inventory_collection
111042025-08-26T21:56:03.661ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/node/id
111052025-08-26T21:56:03.661ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/node/id
111062025-08-26T21:56:03.661ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/node/id
111072025-08-26T21:56:03.661ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/node/id
111082025-08-26T21:56:03.661ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/node/id
111092025-08-26T21:56:03.661ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "1415f4eb-609d-4e7f-964d-e108b3efa677", "content-length": "64", "date": "Tue, 26 Aug 2025 21:56:03 GMT"} })
111102025-08-26T21:56:03.661ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/proxy/status/vars
111112025-08-26T21:56:03.661ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "1c581c83-2b93-48b5-bf9e-82674f2c3e12", "content-length": "64", "date": "Tue, 26 Aug 2025 21:56:03 GMT"} })
111122025-08-26T21:56:03.661ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "1888b175-e05b-4126-9251-4e893dc77fb4", "content-length": "64", "date": "Tue, 26 Aug 2025 21:56:03 GMT"} })
111132025-08-26T21:56:03.661ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "a4bbbb1d-a5e9-48de-83a8-33d16a8634a3", "content-length": "64", "date": "Tue, 26 Aug 2025 21:56:03 GMT"} })
111142025-08-26T21:56:03.661ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/proxy/status/vars
111152025-08-26T21:56:03.661ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/proxy/status/vars
111162025-08-26T21:56:03.661ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/proxy/status/vars
111172025-08-26T21:56:03.661ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "c9b28fe8-7792-4466-a7c3-f54896ae3d9d", "content-length": "64", "date": "Tue, 26 Aug 2025 21:56:03 GMT"} })
111182025-08-26T21:56:03.661ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/proxy/status/vars
111192025-08-26T21:56:03.662ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1832 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
111202025-08-26T21:56:03.662ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1859 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 88d84f37-a4c8-4abc-af1f-27987b0d39d9
111212025-08-26T21:56:03.662ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
111222025-08-26T21:56:03.662ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
111232025-08-26T21:56:03.662ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
111242025-08-26T21:56:03.662ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
111252025-08-26T21:56:03.662ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
111262025-08-26T21:56:03.662ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2006 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
111272025-08-26T21:56:03.662ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:408 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
111282025-08-26T21:56:03.662ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2042 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
111292025-08-26T21:56:03.662ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
111302025-08-26T21:56:03.662ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
111312025-08-26T21:56:03.663ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
111322025-08-26T21:56:03.666ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0
111332025-08-26T21:56:03.666ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
111342025-08-26T21:56:03.699ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "51635840-2c2f-4942-8c60-0a023e5d347c", "content-length": "412637", "date": "Tue, 26 Aug 2025 21:56:03 GMT"} })
111352025-08-26T21:56:03.700ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "b4f5aa25-7ce0-449f-833e-3c9ea3b2b7fe", "content-length": "396788", "date": "Tue, 26 Aug 2025 21:56:03 GMT"} })
111362025-08-26T21:56:03.701ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "11f2f944-34e6-48b1-9e0d-88537f3d7fc5", "content-length": "389642", "date": "Tue, 26 Aug 2025 21:56:03 GMT"} })
111372025-08-26T21:56:03.701ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "9661e0aa-df45-43aa-b7b7-d9c2bb9ca9f8", "content-length": "388374", "date": "Tue, 26 Aug 2025 21:56:03 GMT"} })
111382025-08-26T21:56:03.702ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "71154206-c184-41e1-b96c-3515abbe768b", "content-length": "402598", "date": "Tue, 26 Aug 2025 21:56:03 GMT"} })
111392025-08-26T21:56:03.705ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 1
111402025-08-26T21:56:03.705ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 3
111412025-08-26T21:56:03.705ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 2
111422025-08-26T21:56:03.705ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 4
111432025-08-26T21:56:03.705ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 5
111442025-08-26T21:56:03.737ZDEBGnexus (BackgroundTasks): begin collection from NTP admin (timesync) background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1b]:10123 zone_id = e17ff130-2852-42b2-9cfd-58a3a8b33eb6 (service)
111452025-08-26T21:56:03.737ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None method = GET ntp_admin_url = http://[fd00:1122:3344:101::1b]:10123 uri = http://[fd00:1122:3344:101::1b]:10123/timesync
111462025-08-26T21:56:03.740ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection ntp_admin_url = http://[fd00:1122:3344:101::1b]:10123 result = Ok(Response { url: "http://[fd00:1122:3344:101::1b]:10123/timesync", status: 200, headers: {"content-type": "application/json", "x-request-id": "32174418-1c60-41c5-aa61-765bdc00a319", "content-length": "125", "date": "Tue, 26 Aug 2025 21:56:03 GMT"} })
111472025-08-26T21:56:03.742ZDEBGnexus (BackgroundTasks): begin collection from internal DNS servers background_task = inventory_collection
111482025-08-26T21:56:03.770ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "fbe276ec-7465-4828-980f-c901dfc5fc55", "date": "Tue, 26 Aug 2025 21:56:03 GMT"} })
111492025-08-26T21:56:03.770ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch = Switch0
111502025-08-26T21:56:03.770ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
111512025-08-26T21:56:03.771ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "3e575119-6f82-4e47-b5d9-1fd3ab9cdff6", "content-length": "682", "date": "Tue, 26 Aug 2025 21:56:03 GMT"} })
111522025-08-26T21:56:03.771ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
111532025-08-26T21:56:03.771ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
111542025-08-26T21:56:03.775ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
111552025-08-26T21:56:03.775ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
111562025-08-26T21:56:03.779ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
111572025-08-26T21:56:03.779ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
111582025-08-26T21:56:03.781ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
111592025-08-26T21:56:03.781ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
111602025-08-26T21:56:03.791ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 922.190698ms iteration = 21 status = {}
111612025-08-26T21:56:03.833ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 852fd1a3-de9a-47e8-a667-5e8dab880ad3 (service)
111622025-08-26T21:56:03.833ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:2::1]:5353 method = GET uri = http://[fd00:1122:3344:2::1]:5353/config
111632025-08-26T21:56:03.834ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:2::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "68781275-7cd3-440b-a737-f96f1ffc6f8b", "content-length": "9110", "date": "Tue, 26 Aug 2025 21:56:03 GMT"} })
111642025-08-26T21:56:03.834ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 852fd1a3-de9a-47e8-a667-5e8dab880ad3 (service)
111652025-08-26T21:56:03.835ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = c68f1501-294c-4874-92ca-8fc0546a5870 (service)
111662025-08-26T21:56:03.835ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:1::1]:5353 method = GET uri = http://[fd00:1122:3344:1::1]:5353/config
111672025-08-26T21:56:03.836ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:1::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "6f7cabc7-bfa4-4ac1-8ad9-3dd2812a441c", "content-length": "9110", "date": "Tue, 26 Aug 2025 21:56:03 GMT"} })
111682025-08-26T21:56:03.836ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = c68f1501-294c-4874-92ca-8fc0546a5870 (service)
111692025-08-26T21:56:03.838ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = d741a72f-8745-4723-9e80-e1e72987266b (service)
111702025-08-26T21:56:03.838ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:3::1]:5353 method = GET uri = http://[fd00:1122:3344:3::1]:5353/config
111712025-08-26T21:56:03.839ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:3::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "97cfa5eb-4594-4dd6-91f8-f969702d3e6e", "content-length": "9110", "date": "Tue, 26 Aug 2025 21:56:03 GMT"} })
111722025-08-26T21:56:03.839ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = d741a72f-8745-4723-9e80-e1e72987266b (service)
111732025-08-26T21:56:03.839ZDEBGnexus (BackgroundTasks): finished collection from internal DNS servers background_task = inventory_collection
111742025-08-26T21:56:03.839ZDEBGnexus (BackgroundTasks): finished collection background_task = inventory_collection
111752025-08-26T21:56:03.841ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
111762025-08-26T21:56:03.841ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
111772025-08-26T21:56:03.843ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
111782025-08-26T21:56:03.845ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
111792025-08-26T21:56:03.845ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
111802025-08-26T21:56:03.846ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
111812025-08-26T21:56:04.044ZINFOnexus (BackgroundTasks): inserted inventory collection background_task = inventory_collection collection_id = 0a8628e9-60fa-4f2b-a819-b89586c72d94 file = nexus/db-queries/src/db/datastore/inventory.rs:1614
111822025-08-26T21:56:04.044ZDEBGnexus (BackgroundTasks): inventory collection complete background_task = inventory_collection collection_id = 0a8628e9-60fa-4f2b-a819-b89586c72d94 time_started = 2025-08-26 21:56:03.479288 UTC
111832025-08-26T21:56:04.044ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 14 reason = Dependency
111842025-08-26T21:56:04.044ZDEBGnexus (BackgroundTasks): blueprint planning disabled, doing nothing background_task = blueprint_planner
111852025-08-26T21:56:04.044ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 23 reason = Dependency
111862025-08-26T21:56:04.044ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 10.79\u{b5}s iteration = 14 status = "Disabled"
111872025-08-26T21:56:04.044ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_rendezvous iteration = 5 reason = Dependency
111882025-08-26T21:56:04.044ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
111892025-08-26T21:56:04.044ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
111902025-08-26T21:56:04.044ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 1.203310625s iteration = 2 status = {"collection_id":"0a8628e9-60fa-4f2b-a819-b89586c72d94","time_done":"2025-08-26 21:56:03.839953 UTC","time_started":"2025-08-26 21:56:03.479288 UTC"}
111912025-08-26T21:56:04.045ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
111922025-08-26T21:56:04.045ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
111932025-08-26T21:56:04.047ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
111942025-08-26T21:56:04.047ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
111952025-08-26T21:56:04.051ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
111962025-08-26T21:56:04.051ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
111972025-08-26T21:56:04.051ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
111982025-08-26T21:56:04.052ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Inventory result = Ok(())
111992025-08-26T21:56:04.052ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
112002025-08-26T21:56:04.052ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
112012025-08-26T21:56:04.054ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
112022025-08-26T21:56:04.054ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
112032025-08-26T21:56:04.054ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
112042025-08-26T21:56:04.055ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
112052025-08-26T21:56:04.057ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
112062025-08-26T21:56:04.059ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
112072025-08-26T21:56:04.059ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
112082025-08-26T21:56:04.059ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
112092025-08-26T21:56:04.078ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 34.382425ms iteration = 23 status = {"physical_disks_added":0}
112102025-08-26T21:56:04.104ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
112112025-08-26T21:56:04.105ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
112122025-08-26T21:56:04.107ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
112132025-08-26T21:56:04.110ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
112142025-08-26T21:56:04.110ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
112152025-08-26T21:56:04.110ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
112162025-08-26T21:56:04.112ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
112172025-08-26T21:56:04.114ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
112182025-08-26T21:56:04.114ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
112192025-08-26T21:56:04.115ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
112202025-08-26T21:56:04.117ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
112212025-08-26T21:56:04.118ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
112222025-08-26T21:56:04.120ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
112232025-08-26T21:56:04.122ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
112242025-08-26T21:56:04.123ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
112252025-08-26T21:56:04.123ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
112262025-08-26T21:56:04.125ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
112272025-08-26T21:56:04.127ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
112282025-08-26T21:56:04.127ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
112292025-08-26T21:56:04.127ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
112302025-08-26T21:56:04.130ZINFOnexus (BackgroundTasks): ensured all Crucible datasets present in inventory have database records background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/crucible_dataset.rs:113 num_already_exist = 9 num_inserted = 0 num_not_in_inventory = 0
112312025-08-26T21:56:04.130ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_rendezvous elapsed = 85.889946ms iteration = 5 status = {"blueprint_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","inventory_collection_id":"0a8628e9-60fa-4f2b-a819-b89586c72d94","stats":{"crucible_dataset":{"num_already_exist":9,"num_inserted":0,"num_not_in_inventory":0},"debug_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0}}}
112322025-08-26T21:56:05.352ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 284 local_addr = [fd00:1122:3344:101::c]:42016 method = GET remote_addr = [fd00:1122:3344:101::d]:37550 req_id = 39d58de7-6fff-4d69-b98b-c3836170d959 response_code = 200 uri = /5d0959e5-47a6-4bf3-a52b-57950b9d900d
112332025-08-26T21:56:07.869ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 122 reason = Timeout
112342025-08-26T21:56:07.869ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
112352025-08-26T21:56:07.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
112362025-08-26T21:56:07.871ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
112372025-08-26T21:56:07.874ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
112382025-08-26T21:56:07.874ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
112392025-08-26T21:56:07.874ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
112402025-08-26T21:56:07.876ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
112412025-08-26T21:56:07.876ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.247063ms iteration = 122 status = {"chicken_switches_updated":false}
112422025-08-26T21:56:12.869ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 62 reason = Timeout
112432025-08-26T21:56:12.869ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
112442025-08-26T21:56:12.869ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 123 reason = Timeout
112452025-08-26T21:56:12.869ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
112462025-08-26T21:56:12.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
112472025-08-26T21:56:12.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
112482025-08-26T21:56:12.871ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
112492025-08-26T21:56:12.871ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
112502025-08-26T21:56:12.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
112512025-08-26T21:56:12.871ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 62 reason = Timeout
112522025-08-26T21:56:12.871ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
112532025-08-26T21:56:12.872ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
112542025-08-26T21:56:12.873ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.356718ms iteration = 62 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
112552025-08-26T21:56:12.873ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
112562025-08-26T21:56:12.874ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
112572025-08-26T21:56:12.874ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
112582025-08-26T21:56:12.874ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
112592025-08-26T21:56:12.876ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
112602025-08-26T21:56:12.876ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.253003ms iteration = 123 status = {"chicken_switches_updated":false}
112612025-08-26T21:56:12.876ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
112622025-08-26T21:56:12.876ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
112632025-08-26T21:56:12.877ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
112642025-08-26T21:56:12.878ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
112652025-08-26T21:56:12.879ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
112662025-08-26T21:56:12.880ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
112672025-08-26T21:56:12.883ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
112682025-08-26T21:56:12.883ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
112692025-08-26T21:56:12.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
112702025-08-26T21:56:12.899ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
112712025-08-26T21:56:12.899ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 27.510171ms iteration = 62 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
112722025-08-26T21:56:15.345ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:61737 req_id = ac39f23b-5f55-4eed-a4ae-74017ee463c0 roles = RoleSet { roles: {} } uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?sort_by=id_ascending
112732025-08-26T21:56:15.345ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:61737 req_id = ac39f23b-5f55-4eed-a4ae-74017ee463c0 resource = Database result = Ok(()) uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?sort_by=id_ascending
112742025-08-26T21:56:15.348ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 2730 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:61737 req_id = ac39f23b-5f55-4eed-a4ae-74017ee463c0 response_code = 200 uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?sort_by=id_ascending
112752025-08-26T21:56:15.348ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:61737 req_id = dab6e62e-ebc0-4729-a53e-e169b228bda0 roles = RoleSet { roles: {} } uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMzhjYTg4YS1lZGE1LTQwZGEtOGE4MS1hN2UzMzdhMmM4ZmIifX0%3D
112762025-08-26T21:56:15.348ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:61737 req_id = dab6e62e-ebc0-4729-a53e-e169b228bda0 resource = Database result = Ok(()) uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMzhjYTg4YS1lZGE1LTQwZGEtOGE4MS1hN2UzMzdhMmM4ZmIifX0%3D
112772025-08-26T21:56:15.349ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 1553 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:61737 req_id = dab6e62e-ebc0-4729-a53e-e169b228bda0 response_code = 200 uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMzhjYTg4YS1lZGE1LTQwZGEtOGE4MS1hN2UzMzdhMmM4ZmIifX0%3D
112782025-08-26T21:56:15.352ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 269 local_addr = [fd00:1122:3344:101::c]:42016 method = GET remote_addr = [fd00:1122:3344:101::d]:37550 req_id = 3575a08e-7ae9-4c03-ab78-d03fcd35dbf0 response_code = 200 uri = /5d0959e5-47a6-4bf3-a52b-57950b9d900d
112792025-08-26T21:56:17.868ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 124 reason = Timeout
112802025-08-26T21:56:17.868ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
112812025-08-26T21:56:17.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
112822025-08-26T21:56:17.871ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
112832025-08-26T21:56:17.873ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
112842025-08-26T21:56:17.873ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
112852025-08-26T21:56:17.874ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
112862025-08-26T21:56:17.876ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
112872025-08-26T21:56:17.876ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.387576ms iteration = 124 status = {"chicken_switches_updated":false}
112882025-08-26T21:56:22.869ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 63 reason = Timeout
112892025-08-26T21:56:22.869ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
112902025-08-26T21:56:22.869ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 125 reason = Timeout
112912025-08-26T21:56:22.869ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
112922025-08-26T21:56:22.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
112932025-08-26T21:56:22.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
112942025-08-26T21:56:22.871ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 63 reason = Timeout
112952025-08-26T21:56:22.871ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
112962025-08-26T21:56:22.871ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
112972025-08-26T21:56:22.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
112982025-08-26T21:56:22.872ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
112992025-08-26T21:56:22.872ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
113002025-08-26T21:56:22.873ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
113012025-08-26T21:56:22.873ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.031521ms iteration = 63 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
113022025-08-26T21:56:22.875ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
113032025-08-26T21:56:22.875ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
113042025-08-26T21:56:22.875ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
113052025-08-26T21:56:22.876ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
113062025-08-26T21:56:22.876ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
113072025-08-26T21:56:22.876ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
113082025-08-26T21:56:22.876ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
113092025-08-26T21:56:22.876ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.498199ms iteration = 125 status = {"chicken_switches_updated":false}
113102025-08-26T21:56:22.878ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
113112025-08-26T21:56:22.878ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
113122025-08-26T21:56:22.880ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
113132025-08-26T21:56:22.882ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
113142025-08-26T21:56:22.882ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
113152025-08-26T21:56:22.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
113162025-08-26T21:56:22.895ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
113172025-08-26T21:56:22.895ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 24.038233ms iteration = 63 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
113182025-08-26T21:56:25.352ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 453 local_addr = [fd00:1122:3344:101::c]:42016 method = GET remote_addr = [fd00:1122:3344:101::d]:37550 req_id = 8f3e3de6-8132-4f9d-9940-0c9bdb26ee4c response_code = 200 uri = /5d0959e5-47a6-4bf3-a52b-57950b9d900d
113192025-08-26T21:56:27.869ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 126 reason = Timeout
113202025-08-26T21:56:27.869ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
113212025-08-26T21:56:27.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
113222025-08-26T21:56:27.871ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
113232025-08-26T21:56:27.874ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
113242025-08-26T21:56:27.874ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
113252025-08-26T21:56:27.874ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
113262025-08-26T21:56:27.876ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
113272025-08-26T21:56:27.876ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.162412ms iteration = 126 status = {"chicken_switches_updated":false}
113282025-08-26T21:56:32.832ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 22 reason = Timeout
113292025-08-26T21:56:32.832ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 22 reason = Timeout
113302025-08-26T21:56:32.832ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 22 reason = Timeout
113312025-08-26T21:56:32.832ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
113322025-08-26T21:56:32.832ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
113332025-08-26T21:56:32.832ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
113342025-08-26T21:56:32.832ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
113352025-08-26T21:56:32.835ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1496, tv_nsec: 694401078 } })
113362025-08-26T21:56:32.835ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1496, tv_nsec: 694407289 } })
113372025-08-26T21:56:32.835ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1205
113382025-08-26T21:56:32.835ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1205
113392025-08-26T21:56:32.835ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.441497ms iteration = 22 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
113402025-08-26T21:56:32.868ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 22 reason = Timeout
113412025-08-26T21:56:32.868ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 22 reason = Timeout
113422025-08-26T21:56:32.868ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
113432025-08-26T21:56:32.868ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 14.42\u{b5}s iteration = 22 status = {"disabled":true,"errors":[],"sps":[]}
113442025-08-26T21:56:32.868ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 22 reason = Timeout
113452025-08-26T21:56:32.868ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 22 reason = Timeout
113462025-08-26T21:56:32.869ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
113472025-08-26T21:56:32.869ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
113482025-08-26T21:56:32.869ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
113492025-08-26T21:56:32.869ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 22 reason = Timeout
113502025-08-26T21:56:32.869ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
113512025-08-26T21:56:32.869ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
113522025-08-26T21:56:32.869ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 64 reason = Timeout
113532025-08-26T21:56:32.869ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 22 reason = Timeout
113542025-08-26T21:56:32.869ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
113552025-08-26T21:56:32.869ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
113562025-08-26T21:56:32.869ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 24 reason = Timeout
113572025-08-26T21:56:32.869ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
113582025-08-26T21:56:32.869ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 22 reason = Timeout
113592025-08-26T21:56:32.869ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
113602025-08-26T21:56:32.869ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 22 reason = Timeout
113612025-08-26T21:56:32.869ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
113622025-08-26T21:56:32.869ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 127 reason = Timeout
113632025-08-26T21:56:32.869ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
113642025-08-26T21:56:32.869ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 22 reason = Timeout
113652025-08-26T21:56:32.869ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
113662025-08-26T21:56:32.869ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 22 reason = Timeout
113672025-08-26T21:56:32.869ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
113682025-08-26T21:56:32.869ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 22 reason = Timeout
113692025-08-26T21:56:32.869ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
113702025-08-26T21:56:32.869ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 22 reason = Timeout
113712025-08-26T21:56:32.869ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 22 reason = Timeout
113722025-08-26T21:56:32.869ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
113732025-08-26T21:56:32.869ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
113742025-08-26T21:56:32.869ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 22 reason = Timeout
113752025-08-26T21:56:32.869ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
113762025-08-26T21:56:32.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
113772025-08-26T21:56:32.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
113782025-08-26T21:56:32.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
113792025-08-26T21:56:32.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
113802025-08-26T21:56:32.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
113812025-08-26T21:56:32.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
113822025-08-26T21:56:32.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
113832025-08-26T21:56:32.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
113842025-08-26T21:56:32.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
113852025-08-26T21:56:32.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
113862025-08-26T21:56:32.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
113872025-08-26T21:56:32.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
113882025-08-26T21:56:32.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
113892025-08-26T21:56:32.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
113902025-08-26T21:56:32.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
113912025-08-26T21:56:32.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
113922025-08-26T21:56:32.871ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
113932025-08-26T21:56:32.871ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
113942025-08-26T21:56:32.871ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
113952025-08-26T21:56:32.871ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 64 reason = Timeout
113962025-08-26T21:56:32.871ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
113972025-08-26T21:56:32.871ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
113982025-08-26T21:56:32.871ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
113992025-08-26T21:56:32.871ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
114002025-08-26T21:56:32.871ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
114012025-08-26T21:56:32.871ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
114022025-08-26T21:56:32.871ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
114032025-08-26T21:56:32.871ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
114042025-08-26T21:56:32.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
114052025-08-26T21:56:32.872ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 2.970277ms iteration = 22 status = {"errors":[],"garbage_collect_requested":[]}
114062025-08-26T21:56:32.872ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
114072025-08-26T21:56:32.872ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
114082025-08-26T21:56:32.872ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
114092025-08-26T21:56:32.873ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
114102025-08-26T21:56:32.873ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.86984ms iteration = 64 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
114112025-08-26T21:56:32.874ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1213 zone_address = fd00:1122:3344:101::2
114122025-08-26T21:56:32.874ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
114132025-08-26T21:56:32.874ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "bdc688ec-3f68-4047-be5f-951e17d65823", "content-length": "26", "date": "Tue, 26 Aug 2025 21:56:32 GMT"} })
114142025-08-26T21:56:32.874ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1216 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
114152025-08-26T21:56:32.875ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1213 zone_address = fd00:1122:3344:101::2
114162025-08-26T21:56:32.875ZDEBGnexus (MgsClient): client request background_task = nat_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
114172025-08-26T21:56:32.875ZDEBGnexus (MgsClient): client response background_task = nat_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "5ca1dcd6-a9de-4dd7-ae95-7c11291a64aa", "content-length": "26", "date": "Tue, 26 Aug 2025 21:56:32 GMT"} })
114182025-08-26T21:56:32.875ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1216 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
114192025-08-26T21:56:32.876ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 7.176132ms iteration = 22 status = {"errors":[],"requests_created_ok":[]}
114202025-08-26T21:56:32.876ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
114212025-08-26T21:56:32.876ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
114222025-08-26T21:56:32.876ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
114232025-08-26T21:56:32.876ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
114242025-08-26T21:56:32.877ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
114252025-08-26T21:56:32.877ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
114262025-08-26T21:56:32.877ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
114272025-08-26T21:56:32.877ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
114282025-08-26T21:56:32.877ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
114292025-08-26T21:56:32.877ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
114302025-08-26T21:56:32.877ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
114312025-08-26T21:56:32.877ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
114322025-08-26T21:56:32.877ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
114332025-08-26T21:56:32.877ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
114342025-08-26T21:56:32.877ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
114352025-08-26T21:56:32.877ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
114362025-08-26T21:56:32.877ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
114372025-08-26T21:56:32.878ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
114382025-08-26T21:56:32.878ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
114392025-08-26T21:56:32.878ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
114402025-08-26T21:56:32.878ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
114412025-08-26T21:56:32.878ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
114422025-08-26T21:56:32.878ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
114432025-08-26T21:56:32.879ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
114442025-08-26T21:56:32.879ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
114452025-08-26T21:56:32.879ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
114462025-08-26T21:56:32.879ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
114472025-08-26T21:56:32.879ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
114482025-08-26T21:56:32.879ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
114492025-08-26T21:56:32.879ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1496, tv_nsec: 738871201 } })
114502025-08-26T21:56:32.880ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1205 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
114512025-08-26T21:56:32.880ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
114522025-08-26T21:56:32.880ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 11.52982ms iteration = 127 status = {"chicken_switches_updated":false}
114532025-08-26T21:56:32.880ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
114542025-08-26T21:56:32.881ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
114552025-08-26T21:56:32.881ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
114562025-08-26T21:56:32.881ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
114572025-08-26T21:56:32.881ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
114582025-08-26T21:56:32.881ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
114592025-08-26T21:56:32.881ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
114602025-08-26T21:56:32.881ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
114612025-08-26T21:56:32.882ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
114622025-08-26T21:56:32.883ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:550 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
114632025-08-26T21:56:32.883ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 14.427555ms iteration = 22 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
114642025-08-26T21:56:32.884ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
114652025-08-26T21:56:32.884ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
114662025-08-26T21:56:32.884ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
114672025-08-26T21:56:32.885ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
114682025-08-26T21:56:32.885ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
114692025-08-26T21:56:32.885ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
114702025-08-26T21:56:32.892ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 23.699184ms iteration = 24 status = {"physical_disks_added":0}
114712025-08-26T21:56:32.902ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
114722025-08-26T21:56:32.902ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 31.037869ms iteration = 64 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
114732025-08-26T21:56:32.921ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 52.102645ms iteration = 22 status = {"errors":[],"finish_invoked_ok":[]}
114742025-08-26T21:56:32.923ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
114752025-08-26T21:56:32.924ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
114762025-08-26T21:56:32.925ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
114772025-08-26T21:56:32.926ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
114782025-08-26T21:56:32.928ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
114792025-08-26T21:56:32.928ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
114802025-08-26T21:56:32.928ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
114812025-08-26T21:56:32.928ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
114822025-08-26T21:56:32.928ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
114832025-08-26T21:56:32.929ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
114842025-08-26T21:56:32.929ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
114852025-08-26T21:56:32.929ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
114862025-08-26T21:56:32.929ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
114872025-08-26T21:56:32.929ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
114882025-08-26T21:56:32.930ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
114892025-08-26T21:56:32.930ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 61.314302ms iteration = 22 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
114902025-08-26T21:56:32.930ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 61.620279ms iteration = 22 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
114912025-08-26T21:56:32.931ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
114922025-08-26T21:56:32.931ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
114932025-08-26T21:56:32.932ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
114942025-08-26T21:56:32.932ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
114952025-08-26T21:56:32.933ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
114962025-08-26T21:56:32.934ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:407
114972025-08-26T21:56:32.934ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 65.448075ms iteration = 22 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
114982025-08-26T21:56:32.938ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
114992025-08-26T21:56:32.938ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
115002025-08-26T21:56:32.939ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
115012025-08-26T21:56:32.940ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
115022025-08-26T21:56:32.941ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
115032025-08-26T21:56:32.944ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: dd0c655a-95c2-48fc-b4c3-9733ed4b0faa, lookup_type: ById(dd0c655a-95c2-48fc-b4c3-9733ed4b0faa) } result = Ok(())
115042025-08-26T21:56:32.944ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
115052025-08-26T21:56:32.944ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
115062025-08-26T21:56:32.946ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 77.374144ms iteration = 22 status = 0
115072025-08-26T21:56:32.964ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
115082025-08-26T21:56:32.964ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
115092025-08-26T21:56:32.964ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 95.990914ms iteration = 22 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
115102025-08-26T21:56:32.965ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 96.517106ms iteration = 22 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
115112025-08-26T21:56:32.972ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
115122025-08-26T21:56:32.973ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
115132025-08-26T21:56:32.974ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
115142025-08-26T21:56:32.975ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
115152025-08-26T21:56:32.976ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
115162025-08-26T21:56:32.976ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
115172025-08-26T21:56:32.976ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
115182025-08-26T21:56:32.976ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
115192025-08-26T21:56:32.977ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 145.450879ms iteration = 22 status = null
115202025-08-26T21:56:32.978ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
115212025-08-26T21:56:32.980ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
115222025-08-26T21:56:32.980ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
115232025-08-26T21:56:32.980ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
115242025-08-26T21:56:33.070ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 237.676198ms iteration = 22 status = {}
115252025-08-26T21:56:33.071ZINFOnexus (BackgroundTasks): VPC route manager sled dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
115262025-08-26T21:56:33.071ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
115272025-08-26T21:56:33.072ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "30a07612-46a5-49db-a7c9-f8b8114b3da0", "content-length": "372", "date": "Tue, 26 Aug 2025 21:56:33 GMT"} })
115282025-08-26T21:56:33.072ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
115292025-08-26T21:56:33.072ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
115302025-08-26T21:56:33.076ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
115312025-08-26T21:56:33.077ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "674bb6b8-b477-4530-aa91-23682ba06346", "content-length": "691", "date": "Tue, 26 Aug 2025 21:56:33 GMT"} })
115322025-08-26T21:56:33.077ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
115332025-08-26T21:56:33.077ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
115342025-08-26T21:56:33.077ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
115352025-08-26T21:56:33.078ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 209.81378ms iteration = 22 status = {}
115362025-08-26T21:56:33.080ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1213 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 zone_address = fd00:1122:3344:101::2
115372025-08-26T21:56:33.080ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
115382025-08-26T21:56:33.080ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "d0397e6e-0fee-40fc-8e7a-072bd741a055", "content-length": "26", "date": "Tue, 26 Aug 2025 21:56:32 GMT"} })
115392025-08-26T21:56:33.080ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1216 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
115402025-08-26T21:56:33.082ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
115412025-08-26T21:56:33.121ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
115422025-08-26T21:56:33.121ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
115432025-08-26T21:56:33.121ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "4d44ac75-9167-40a5-9ff5-5ad85072edf1", "date": "Tue, 26 Aug 2025 21:56:33 GMT"} })
115442025-08-26T21:56:33.121ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
115452025-08-26T21:56:33.122ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
115462025-08-26T21:56:33.124ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
115472025-08-26T21:56:33.124ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
115482025-08-26T21:56:33.127ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
115492025-08-26T21:56:33.127ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
115502025-08-26T21:56:33.131ZINFOnexus (BackgroundTasks): VPC route manager sled dd0c655a-95c2-48fc-b4c3-9733ed4b0faa push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
115512025-08-26T21:56:33.131ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
115522025-08-26T21:56:33.131ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "a398dd34-42e9-4f18-b46a-05bf332a9357", "date": "Tue, 26 Aug 2025 21:56:32 GMT"} })
115532025-08-26T21:56:33.133ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 264.689976ms iteration = 22 status = {}
115542025-08-26T21:56:33.174ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
115552025-08-26T21:56:33.175ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
115562025-08-26T21:56:33.177ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: d24c5c55-3a21-4a91-aeb4-45362e822df6, rack_id: e37a46f2-f789-4ca4-9841-f8c6dd705964, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(88d84f37-a4c8-4abc-af1f-27987b0d39d9) } rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0
115572025-08-26T21:56:33.177ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
115582025-08-26T21:56:33.177ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
115592025-08-26T21:56:33.188ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1832 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
115602025-08-26T21:56:33.188ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1859 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 88d84f37-a4c8-4abc-af1f-27987b0d39d9
115612025-08-26T21:56:33.189ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
115622025-08-26T21:56:33.189ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
115632025-08-26T21:56:33.189ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
115642025-08-26T21:56:33.189ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
115652025-08-26T21:56:33.189ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
115662025-08-26T21:56:33.189ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2006 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
115672025-08-26T21:56:33.189ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:408 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
115682025-08-26T21:56:33.189ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2042 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
115692025-08-26T21:56:33.189ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
115702025-08-26T21:56:33.189ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
115712025-08-26T21:56:33.189ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
115722025-08-26T21:56:33.192ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0
115732025-08-26T21:56:33.192ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
115742025-08-26T21:56:33.302ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "bfc9622a-aaa3-4ed9-88e6-a487d16c4bbe", "date": "Tue, 26 Aug 2025 21:56:33 GMT"} })
115752025-08-26T21:56:33.302ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch = Switch0
115762025-08-26T21:56:33.302ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
115772025-08-26T21:56:33.302ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "28055efd-d6ac-454a-a52c-5ad997fc8c86", "content-length": "682", "date": "Tue, 26 Aug 2025 21:56:33 GMT"} })
115782025-08-26T21:56:33.302ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
115792025-08-26T21:56:33.302ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
115802025-08-26T21:56:33.305ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
115812025-08-26T21:56:33.305ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
115822025-08-26T21:56:33.309ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
115832025-08-26T21:56:33.309ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
115842025-08-26T21:56:33.311ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
115852025-08-26T21:56:33.312ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
115862025-08-26T21:56:33.319ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 450.668979ms iteration = 22 status = {}
115872025-08-26T21:56:35.351ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 293 local_addr = [fd00:1122:3344:101::c]:42016 method = GET remote_addr = [fd00:1122:3344:101::d]:37550 req_id = 9988293f-24f2-4d20-a73b-fb10c853c5b2 response_code = 200 uri = /5d0959e5-47a6-4bf3-a52b-57950b9d900d
115882025-08-26T21:56:37.868ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 128 reason = Timeout
115892025-08-26T21:56:37.868ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
115902025-08-26T21:56:37.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
115912025-08-26T21:56:37.870ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
115922025-08-26T21:56:37.873ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
115932025-08-26T21:56:37.873ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
115942025-08-26T21:56:37.873ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
115952025-08-26T21:56:37.875ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
115962025-08-26T21:56:37.875ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 6.839054ms iteration = 128 status = {"chicken_switches_updated":false}
115972025-08-26T21:56:42.868ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 65 reason = Timeout
115982025-08-26T21:56:42.868ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 129 reason = Timeout
115992025-08-26T21:56:42.868ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
116002025-08-26T21:56:42.868ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
116012025-08-26T21:56:42.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
116022025-08-26T21:56:42.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
116032025-08-26T21:56:42.871ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
116042025-08-26T21:56:42.872ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 65 reason = Timeout
116052025-08-26T21:56:42.872ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
116062025-08-26T21:56:42.872ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
116072025-08-26T21:56:42.873ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
116082025-08-26T21:56:42.873ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
116092025-08-26T21:56:42.873ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
116102025-08-26T21:56:42.874ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
116112025-08-26T21:56:42.875ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
116122025-08-26T21:56:42.875ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 6.415945ms iteration = 129 status = {"chicken_switches_updated":false}
116132025-08-26T21:56:42.876ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
116142025-08-26T21:56:42.876ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
116152025-08-26T21:56:42.876ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
116162025-08-26T21:56:42.882ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
116172025-08-26T21:56:42.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
116182025-08-26T21:56:42.884ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
116192025-08-26T21:56:42.887ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
116202025-08-26T21:56:42.887ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
116212025-08-26T21:56:42.887ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
116222025-08-26T21:56:42.905ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
116232025-08-26T21:56:42.905ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 33.385642ms iteration = 65 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
116242025-08-26T21:56:42.913ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
116252025-08-26T21:56:42.913ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
116262025-08-26T21:56:42.915ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 46.454797ms iteration = 65 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
116272025-08-26T21:56:45.351ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 283 local_addr = [fd00:1122:3344:101::c]:42016 method = GET remote_addr = [fd00:1122:3344:101::d]:37550 req_id = e5f08c99-037b-4c17-9f6d-ea4afe71c645 response_code = 200 uri = /5d0959e5-47a6-4bf3-a52b-57950b9d900d
116282025-08-26T21:56:47.868ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 130 reason = Timeout
116292025-08-26T21:56:47.868ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
116302025-08-26T21:56:47.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
116312025-08-26T21:56:47.871ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
116322025-08-26T21:56:47.874ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
116332025-08-26T21:56:47.874ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
116342025-08-26T21:56:47.874ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
116352025-08-26T21:56:47.877ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
116362025-08-26T21:56:47.877ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 8.234835ms iteration = 130 status = {"chicken_switches_updated":false}
116372025-08-26T21:56:52.869ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 66 reason = Timeout
116382025-08-26T21:56:52.869ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 131 reason = Timeout
116392025-08-26T21:56:52.869ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
116402025-08-26T21:56:52.869ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
116412025-08-26T21:56:52.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
116422025-08-26T21:56:52.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
116432025-08-26T21:56:52.871ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
116442025-08-26T21:56:52.871ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
116452025-08-26T21:56:52.871ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 66 reason = Timeout
116462025-08-26T21:56:52.871ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
116472025-08-26T21:56:52.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
116482025-08-26T21:56:52.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
116492025-08-26T21:56:52.873ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.071602ms iteration = 66 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
116502025-08-26T21:56:52.873ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
116512025-08-26T21:56:52.874ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
116522025-08-26T21:56:52.874ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
116532025-08-26T21:56:52.874ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
116542025-08-26T21:56:52.876ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
116552025-08-26T21:56:52.876ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.005568ms iteration = 131 status = {"chicken_switches_updated":false}
116562025-08-26T21:56:52.876ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
116572025-08-26T21:56:52.876ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
116582025-08-26T21:56:52.876ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
116592025-08-26T21:56:52.878ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
116602025-08-26T21:56:52.878ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
116612025-08-26T21:56:52.879ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
116622025-08-26T21:56:52.882ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
116632025-08-26T21:56:52.882ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
116642025-08-26T21:56:52.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
116652025-08-26T21:56:52.899ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
116662025-08-26T21:56:52.899ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 28.38453ms iteration = 66 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
116672025-08-26T21:56:55.351ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 294 local_addr = [fd00:1122:3344:101::c]:42016 method = GET remote_addr = [fd00:1122:3344:101::d]:37550 req_id = fc2dffbe-1f64-4f93-972c-0cb94ce9ebd1 response_code = 200 uri = /5d0959e5-47a6-4bf3-a52b-57950b9d900d
116682025-08-26T21:56:57.868ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 132 reason = Timeout
116692025-08-26T21:56:57.868ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
116702025-08-26T21:56:57.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
116712025-08-26T21:56:57.871ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
116722025-08-26T21:56:57.874ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
116732025-08-26T21:56:57.874ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
116742025-08-26T21:56:57.874ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
116752025-08-26T21:56:57.876ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
116762025-08-26T21:56:57.876ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.729354ms iteration = 132 status = {"chicken_switches_updated":false}
116772025-08-26T21:57:02.833ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 12 reason = Timeout
116782025-08-26T21:57:02.833ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 12 reason = Timeout
116792025-08-26T21:57:02.833ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 23 reason = Timeout
116802025-08-26T21:57:02.833ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 14 reason = Timeout
116812025-08-26T21:57:02.833ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
116822025-08-26T21:57:02.833ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 23 reason = Timeout
116832025-08-26T21:57:02.833ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
116842025-08-26T21:57:02.833ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 12 reason = Timeout
116852025-08-26T21:57:02.833ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 23 reason = Timeout
116862025-08-26T21:57:02.833ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 12 reason = Timeout
116872025-08-26T21:57:02.833ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-08-26 21:47:02.833073339 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
116882025-08-26T21:57:02.833ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
116892025-08-26T21:57:02.833ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
116902025-08-26T21:57:02.833ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 14 reason = Timeout
116912025-08-26T21:57:02.833ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 12 reason = Timeout
116922025-08-26T21:57:02.833ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
116932025-08-26T21:57:02.833ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
116942025-08-26T21:57:02.833ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 12 reason = Timeout
116952025-08-26T21:57:02.833ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
116962025-08-26T21:57:02.833ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
116972025-08-26T21:57:02.833ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
116982025-08-26T21:57:02.833ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
116992025-08-26T21:57:02.833ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
117002025-08-26T21:57:02.833ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
117012025-08-26T21:57:02.833ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
117022025-08-26T21:57:02.833ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
117032025-08-26T21:57:02.833ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("60abf731-31e1-4111-a3e2-65da89eaed87.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("84c849eb-0949-41d9-ad02-9a5509144564.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("84c849eb-0949-41d9-ad02-9a5509144564.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::8)) }], valid_until: Instant { tv_sec: 1526, tv_nsec: 692638399 } })
117042025-08-26T21:57:02.833ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("852fd1a3-de9a-47e8-a667-5e8dab880ad3.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("c68f1501-294c-4874-92ca-8fc0546a5870.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("d741a72f-8745-4723-9e80-e1e72987266b.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("d741a72f-8745-4723-9e80-e1e72987266b.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:3::1)) }], valid_until: Instant { tv_sec: 1526, tv_nsec: 692799993 } })
117052025-08-26T21:57:02.834ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
117062025-08-26T21:57:02.834ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.32008ms iteration = 12 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
117072025-08-26T21:57:02.834ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } dns_group = internal
117082025-08-26T21:57:02.834ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.376971ms iteration = 12 status = {"addresses":["[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:3::1]:5353"]}
117092025-08-26T21:57:02.834ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
117102025-08-26T21:57:02.834ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
117112025-08-26T21:57:02.834ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
117122025-08-26T21:57:02.835ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
117132025-08-26T21:57:02.835ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 13 reason = Timeout
117142025-08-26T21:57:02.835ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:91 reason = blueprint disabled target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8
117152025-08-26T21:57:02.835ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 12.95\u{b5}s iteration = 13 status = {"enabled":false,"target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8"}
117162025-08-26T21:57:02.835ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1526, tv_nsec: 694805158 } })
117172025-08-26T21:57:02.835ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
117182025-08-26T21:57:02.836ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-08-26 21:47:02.833073339 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
117192025-08-26T21:57:02.836ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 3.08651ms iteration = 12 status = {"expiration":"2025-08-26T21:47:02.833073339Z","pruned":[]}
117202025-08-26T21:57:02.836ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1205
117212025-08-26T21:57:02.836ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
117222025-08-26T21:57:02.836ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1526, tv_nsec: 695591655 } })
117232025-08-26T21:57:02.836ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1205
117242025-08-26T21:57:02.837ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 4.112003ms iteration = 23 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
117252025-08-26T21:57:02.837ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
117262025-08-26T21:57:02.838ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
117272025-08-26T21:57:02.839ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
117282025-08-26T21:57:02.839ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
117292025-08-26T21:57:02.840ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
117302025-08-26T21:57:02.841ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
117312025-08-26T21:57:02.842ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
117322025-08-26T21:57:02.842ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
117332025-08-26T21:57:02.842ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
117342025-08-26T21:57:02.842ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
117352025-08-26T21:57:02.842ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
117362025-08-26T21:57:02.842ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
117372025-08-26T21:57:02.842ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
117382025-08-26T21:57:02.842ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
117392025-08-26T21:57:02.842ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
117402025-08-26T21:57:02.842ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
117412025-08-26T21:57:02.842ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
117422025-08-26T21:57:02.842ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
117432025-08-26T21:57:02.843ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
117442025-08-26T21:57:02.843ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
117452025-08-26T21:57:02.843ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
117462025-08-26T21:57:02.847ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
117472025-08-26T21:57:02.847ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
117482025-08-26T21:57:02.847ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
117492025-08-26T21:57:02.848ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
117502025-08-26T21:57:02.848ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
117512025-08-26T21:57:02.848ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
117522025-08-26T21:57:02.848ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
117532025-08-26T21:57:02.848ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
117542025-08-26T21:57:02.848ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 8e720f02-dde8-4423-a6e3-91698373805a dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
117552025-08-26T21:57:02.848ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
117562025-08-26T21:57:02.849ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
117572025-08-26T21:57:02.849ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
117582025-08-26T21:57:02.849ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
117592025-08-26T21:57:02.850ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
117602025-08-26T21:57:02.851ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
117612025-08-26T21:57:02.851ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
117622025-08-26T21:57:02.852ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
117632025-08-26T21:57:02.853ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
117642025-08-26T21:57:02.853ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
117652025-08-26T21:57:02.854ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
117662025-08-26T21:57:02.854ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
117672025-08-26T21:57:02.854ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
117682025-08-26T21:57:02.854ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
117692025-08-26T21:57:02.856ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 8e720f02-dde8-4423-a6e3-91698373805a dns_zone_name = control-plane.oxide.internal found_so_far = 59 version = 1
117702025-08-26T21:57:02.856ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
117712025-08-26T21:57:02.856ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2025-08-26 21:46:04.065217 UTC dns_group = internal generation = 1
117722025-08-26T21:57:02.856ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 22.987438ms iteration = 12 status = {"generation":1}
117732025-08-26T21:57:02.860ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 15 reason = Timeout
117742025-08-26T21:57:02.860ZDEBGnexus (BackgroundTasks): blueprint planning disabled, doing nothing background_task = blueprint_planner
117752025-08-26T21:57:02.860ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 9.88\u{b5}s iteration = 15 status = "Disabled"
117762025-08-26T21:57:02.860ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 12 reason = Timeout
117772025-08-26T21:57:02.860ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 12 reason = Timeout
117782025-08-26T21:57:02.860ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:53
117792025-08-26T21:57:02.860ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
117802025-08-26T21:57:02.860ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
117812025-08-26T21:57:02.860ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 12 reason = Timeout
117822025-08-26T21:57:02.860ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
117832025-08-26T21:57:02.860ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
117842025-08-26T21:57:02.860ZDEBGnexus (BackgroundTasks): activating background_task = webhook_deliverator iteration = 12 reason = Timeout
117852025-08-26T21:57:02.860ZDEBGnexus (BackgroundTasks): activating background_task = alert_dispatcher iteration = 12 reason = Timeout
117862025-08-26T21:57:02.860ZDEBGnexus (BackgroundTasks): roles background_task = webhook_deliverator roles = RoleSet { roles: {} }
117872025-08-26T21:57:02.860ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
117882025-08-26T21:57:02.860ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
117892025-08-26T21:57:02.860ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 12 reason = Timeout
117902025-08-26T21:57:02.860ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
117912025-08-26T21:57:02.860ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
117922025-08-26T21:57:02.860ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = alert_dispatcher resource = Database result = Ok(())
117932025-08-26T21:57:02.860ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = webhook_deliverator resource = Database result = Ok(())
117942025-08-26T21:57:02.860ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
117952025-08-26T21:57:02.861ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
117962025-08-26T21:57:02.863ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 2.774202ms iteration = 12 status = {"errors":[],"found_port_ok":[]}
117972025-08-26T21:57:02.863ZINFOnexus (BackgroundTasks): all webhook delivery tasks completed background_task = webhook_deliverator file = nexus/src/app/background/tasks/webhook_deliverator.rs:204 num_receivers = 0
117982025-08-26T21:57:02.863ZDEBGnexus (BackgroundTasks): activation complete background_task = webhook_deliverator elapsed = 3.006668ms iteration = 12 status = {"by_rx":{},"error":null}
117992025-08-26T21:57:02.864ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
118002025-08-26T21:57:02.864ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
118012025-08-26T21:57:02.864ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
118022025-08-26T21:57:02.864ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
118032025-08-26T21:57:02.865ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = alert_dispatcher resource = Database result = Ok(())
118042025-08-26T21:57:02.867ZDEBGnexus (BackgroundTasks): activation complete background_task = alert_dispatcher elapsed = 7.089299ms iteration = 12 status = {"dispatched":[],"errors":[],"glob_version":"183.0.0","globs_reprocessed":{},"no_receivers":[]}
118052025-08-26T21:57:02.869ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 23 reason = Timeout
118062025-08-26T21:57:02.869ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 23 reason = Timeout
118072025-08-26T21:57:02.869ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 13.011\u{b5}s iteration = 23 status = {"disabled":true,"errors":[],"sps":[]}
118082025-08-26T21:57:02.869ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 23 reason = Timeout
118092025-08-26T21:57:02.869ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 25 reason = Timeout
118102025-08-26T21:57:02.869ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
118112025-08-26T21:57:02.869ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
118122025-08-26T21:57:02.869ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
118132025-08-26T21:57:02.869ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 23 reason = Timeout
118142025-08-26T21:57:02.869ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 23 reason = Timeout
118152025-08-26T21:57:02.869ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
118162025-08-26T21:57:02.869ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
118172025-08-26T21:57:02.869ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
118182025-08-26T21:57:02.869ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 23 reason = Timeout
118192025-08-26T21:57:02.869ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
118202025-08-26T21:57:02.869ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 23 reason = Timeout
118212025-08-26T21:57:02.869ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
118222025-08-26T21:57:02.869ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 67 reason = Timeout
118232025-08-26T21:57:02.869ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
118242025-08-26T21:57:02.869ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 23 reason = Timeout
118252025-08-26T21:57:02.869ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
118262025-08-26T21:57:02.869ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 23 reason = Timeout
118272025-08-26T21:57:02.869ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
118282025-08-26T21:57:02.869ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 23 reason = Timeout
118292025-08-26T21:57:02.869ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
118302025-08-26T21:57:02.869ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 23 reason = Timeout
118312025-08-26T21:57:02.869ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
118322025-08-26T21:57:02.869ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 23 reason = Timeout
118332025-08-26T21:57:02.869ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
118342025-08-26T21:57:02.869ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
118352025-08-26T21:57:02.869ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 133 reason = Timeout
118362025-08-26T21:57:02.869ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
118372025-08-26T21:57:02.869ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 23 reason = Timeout
118382025-08-26T21:57:02.869ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
118392025-08-26T21:57:02.869ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 23 reason = Timeout
118402025-08-26T21:57:02.869ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
118412025-08-26T21:57:02.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
118422025-08-26T21:57:02.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
118432025-08-26T21:57:02.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
118442025-08-26T21:57:02.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
118452025-08-26T21:57:02.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
118462025-08-26T21:57:02.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
118472025-08-26T21:57:02.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
118482025-08-26T21:57:02.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
118492025-08-26T21:57:02.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
118502025-08-26T21:57:02.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
118512025-08-26T21:57:02.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
118522025-08-26T21:57:02.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
118532025-08-26T21:57:02.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
118542025-08-26T21:57:02.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
118552025-08-26T21:57:02.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
118562025-08-26T21:57:02.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
118572025-08-26T21:57:02.871ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
118582025-08-26T21:57:02.872ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 67 reason = Timeout
118592025-08-26T21:57:02.872ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
118602025-08-26T21:57:02.872ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
118612025-08-26T21:57:02.874ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
118622025-08-26T21:57:02.874ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
118632025-08-26T21:57:02.874ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
118642025-08-26T21:57:02.876ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
118652025-08-26T21:57:02.876ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
118662025-08-26T21:57:02.877ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
118672025-08-26T21:57:02.877ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
118682025-08-26T21:57:02.878ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
118692025-08-26T21:57:02.880ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
118702025-08-26T21:57:02.882ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
118712025-08-26T21:57:02.882ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
118722025-08-26T21:57:02.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
118732025-08-26T21:57:02.882ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 13.34524ms iteration = 23 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
118742025-08-26T21:57:02.884ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
118752025-08-26T21:57:02.884ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
118762025-08-26T21:57:02.884ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
118772025-08-26T21:57:02.885ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
118782025-08-26T21:57:02.885ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
118792025-08-26T21:57:02.909ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
118802025-08-26T21:57:02.909ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = dfb404ab-acb8-43de-afe8-b429af0c161d dns_zone_name = oxide.test found_so_far = 0 version = 2
118812025-08-26T21:57:02.909ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
118822025-08-26T21:57:02.909ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
118832025-08-26T21:57:02.915ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
118842025-08-26T21:57:02.918ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
118852025-08-26T21:57:02.920ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
118862025-08-26T21:57:02.920ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
118872025-08-26T21:57:02.920ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
118882025-08-26T21:57:02.923ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 53.623489ms iteration = 23 status = {"errors":[],"finish_invoked_ok":[]}
118892025-08-26T21:57:02.925ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
118902025-08-26T21:57:02.925ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
118912025-08-26T21:57:02.926ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
118922025-08-26T21:57:02.928ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
118932025-08-26T21:57:02.928ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:170 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
118942025-08-26T21:57:02.928ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 68.028063ms iteration = 12 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
118952025-08-26T21:57:02.933ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
118962025-08-26T21:57:02.933ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
118972025-08-26T21:57:02.933ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
118982025-08-26T21:57:02.934ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
118992025-08-26T21:57:02.934ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = dfb404ab-acb8-43de-afe8-b429af0c161d dns_zone_name = oxide.test found_so_far = 4 version = 2
119002025-08-26T21:57:02.934ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
119012025-08-26T21:57:02.934ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2025-08-26 21:46:05.296205 UTC dns_group = external generation = 2
119022025-08-26T21:57:02.934ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 101.764634ms iteration = 12 status = {"generation":2}
119032025-08-26T21:57:02.935ZINFOnexus (BackgroundTasks): found Silo/DNS/TLS config (changed) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-08-26T21:45:59.866217Z, time_modified: 2025-08-26T21:45:59.866217Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 9396aa55-de6f-4d6f-96db-d28e1c76fa9c, db_silo: Silo { identity: SiloIdentity { id: 9396aa55-de6f-4d6f-96db-d28e1c76fa9c, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-08-26T21:46:05.245521Z, time_modified: 2025-08-26T21:46:05.245521Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "e8529d720e4b0de099753dcdb373abb870af04baacd7c7dd850f5254858081af" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 9396aa55-de6f-4d6f-96db-d28e1c76fa9c, db_silo: Silo { identity: SiloIdentity { id: 9396aa55-de6f-4d6f-96db-d28e1c76fa9c, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-08-26T21:46:05.245521Z, time_modified: 2025-08-26T21:46:05.245521Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "e8529d720e4b0de099753dcdb373abb870af04baacd7c7dd850f5254858081af" }] }) } file = nexus/src/app/background/tasks/external_endpoints.rs:94
119042025-08-26T21:57:02.935ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 102.210474ms iteration = 12 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"9396aa55-de6f-4d6f-96db-d28e1c76fa9c","tls_certs":["e8529d720e4b0de099753dcdb373abb870af04baacd7c7dd850f5254858081af"]}},"default_endpoint":{"silo_id":"9396aa55-de6f-4d6f-96db-d28e1c76fa9c","tls_certs":["e8529d720e4b0de099753dcdb373abb870af04baacd7c7dd850f5254858081af"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
119052025-08-26T21:57:02.936ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 66.811836ms iteration = 23 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
119062025-08-26T21:57:02.937ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
119072025-08-26T21:57:02.938ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
119082025-08-26T21:57:02.939ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 70.336255ms iteration = 23 status = {"errors":[],"requests_created_ok":[]}
119092025-08-26T21:57:02.939ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
119102025-08-26T21:57:02.939ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
119112025-08-26T21:57:02.939ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
119122025-08-26T21:57:02.939ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:138 found = 0 not_ready_to_be_deleted = 0
119132025-08-26T21:57:02.939ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
119142025-08-26T21:57:02.939ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 79.561823ms iteration = 12 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
119152025-08-26T21:57:02.940ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
119162025-08-26T21:57:02.940ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
119172025-08-26T21:57:02.941ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
119182025-08-26T21:57:02.942ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
119192025-08-26T21:57:02.943ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 73.748153ms iteration = 23 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
119202025-08-26T21:57:02.944ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
119212025-08-26T21:57:02.944ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
119222025-08-26T21:57:02.944ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
119232025-08-26T21:57:02.945ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
119242025-08-26T21:57:02.946ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
119252025-08-26T21:57:02.946ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 74.825547ms iteration = 67 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
119262025-08-26T21:57:02.947ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
119272025-08-26T21:57:02.947ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
119282025-08-26T21:57:02.947ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
119292025-08-26T21:57:02.947ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
119302025-08-26T21:57:02.947ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
119312025-08-26T21:57:02.947ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
119322025-08-26T21:57:02.947ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
119332025-08-26T21:57:02.947ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
119342025-08-26T21:57:02.949ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
119352025-08-26T21:57:02.949ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
119362025-08-26T21:57:02.949ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
119372025-08-26T21:57:02.949ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 80.121336ms iteration = 67 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
119382025-08-26T21:57:02.949ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
119392025-08-26T21:57:02.949ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
119402025-08-26T21:57:02.949ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
119412025-08-26T21:57:02.949ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
119422025-08-26T21:57:02.949ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
119432025-08-26T21:57:02.950ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
119442025-08-26T21:57:02.950ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
119452025-08-26T21:57:02.950ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
119462025-08-26T21:57:02.951ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
119472025-08-26T21:57:02.952ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
119482025-08-26T21:57:02.952ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 82.51057ms iteration = 133 status = {"chicken_switches_updated":false}
119492025-08-26T21:57:02.952ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1526, tv_nsec: 811018957 } })
119502025-08-26T21:57:02.952ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1205 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
119512025-08-26T21:57:02.952ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
119522025-08-26T21:57:02.953ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
119532025-08-26T21:57:02.953ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 83.934412ms iteration = 23 status = {"errors":[],"garbage_collect_requested":[]}
119542025-08-26T21:57:02.953ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
119552025-08-26T21:57:02.954ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
119562025-08-26T21:57:02.954ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
119572025-08-26T21:57:02.954ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
119582025-08-26T21:57:02.956ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
119592025-08-26T21:57:02.956ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
119602025-08-26T21:57:02.956ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
119612025-08-26T21:57:02.985ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
119622025-08-26T21:57:02.985ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
119632025-08-26T21:57:02.985ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 116.22878ms iteration = 23 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
119642025-08-26T21:57:02.994ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
119652025-08-26T21:57:02.995ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
119662025-08-26T21:57:02.996ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
119672025-08-26T21:57:02.997ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1213 zone_address = fd00:1122:3344:101::2
119682025-08-26T21:57:02.997ZDEBGnexus (MgsClient): client request background_task = nat_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
119692025-08-26T21:57:02.998ZDEBGnexus (MgsClient): client response background_task = nat_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "3fbc2fd3-f7c8-4b08-87a7-9697a8d133fb", "content-length": "26", "date": "Tue, 26 Aug 2025 21:57:02 GMT"} })
119702025-08-26T21:57:02.998ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1216 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
119712025-08-26T21:57:02.998ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
119722025-08-26T21:57:02.998ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
119732025-08-26T21:57:02.999ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
119742025-08-26T21:57:02.999ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
119752025-08-26T21:57:03.001ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
119762025-08-26T21:57:03.001ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
119772025-08-26T21:57:03.004ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
119782025-08-26T21:57:03.004ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
119792025-08-26T21:57:03.006ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
119802025-08-26T21:57:03.009ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: dd0c655a-95c2-48fc-b4c3-9733ed4b0faa, lookup_type: ById(dd0c655a-95c2-48fc-b4c3-9733ed4b0faa) } result = Ok(())
119812025-08-26T21:57:03.009ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
119822025-08-26T21:57:03.009ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
119832025-08-26T21:57:03.011ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 142.41266ms iteration = 23 status = 0
119842025-08-26T21:57:03.021ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
119852025-08-26T21:57:03.021ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
119862025-08-26T21:57:03.038ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
119872025-08-26T21:57:03.039ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
119882025-08-26T21:57:03.041ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:407
119892025-08-26T21:57:03.041ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 171.971797ms iteration = 23 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
119902025-08-26T21:57:03.053ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
119912025-08-26T21:57:03.087ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 217.642986ms iteration = 25 status = {"physical_disks_added":0}
119922025-08-26T21:57:03.158ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1213 zone_address = fd00:1122:3344:101::2
119932025-08-26T21:57:03.158ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
119942025-08-26T21:57:03.159ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "1aed81d6-4a6f-4d3e-ab5b-ab96bc773c6c", "content-length": "26", "date": "Tue, 26 Aug 2025 21:57:03 GMT"} })
119952025-08-26T21:57:03.159ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1216 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
119962025-08-26T21:57:03.162ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
119972025-08-26T21:57:03.168ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
119982025-08-26T21:57:03.169ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
119992025-08-26T21:57:03.171ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
120002025-08-26T21:57:03.173ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
120012025-08-26T21:57:03.173ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
120022025-08-26T21:57:03.173ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
120032025-08-26T21:57:03.174ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
120042025-08-26T21:57:03.175ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
120052025-08-26T21:57:03.176ZINFOnexus (BackgroundTasks): instance reincarnation completed successfully background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:113 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0
120062025-08-26T21:57:03.176ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 315.868971ms iteration = 12 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
120072025-08-26T21:57:03.177ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
120082025-08-26T21:57:03.177ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
120092025-08-26T21:57:03.178ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
120102025-08-26T21:57:03.182ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = instance_watcher body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vmms/b931750f-371f-4349-9ab8-e4868c2650e3/state
120112025-08-26T21:57:03.185ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1213 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 zone_address = fd00:1122:3344:101::2
120122025-08-26T21:57:03.185ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
120132025-08-26T21:57:03.185ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "22062806-0476-4adc-8016-8db385a00223", "content-length": "26", "date": "Tue, 26 Aug 2025 21:57:02 GMT"} })
120142025-08-26T21:57:03.185ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1216 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
120152025-08-26T21:57:03.185ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = instance_watcher result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vmms/b931750f-371f-4349-9ab8-e4868c2650e3/state", status: 200, headers: {"content-type": "application/json", "x-request-id": "25205213-4ae2-4141-96d0-352cfbecfb13", "content-length": "130", "date": "Tue, 26 Aug 2025 21:57:02 GMT"} })
120162025-08-26T21:57:03.187ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
120172025-08-26T21:57:03.188ZDEBGnexus (BackgroundTasks): updating instance state background_task = instance_watcher instance_id = 5f8711b0-0cf9-4ef2-aaaa-3cc9f32ebe3c sled_id = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa state = SledVmmState { vmm_state: VmmRuntimeState { state: Running, gen: Generation(4), time_updated: 2025-08-26T21:57:01.346616291Z }, migration_in: None, migration_out: None } vmm_id = b931750f-371f-4349-9ab8-e4868c2650e3
120182025-08-26T21:57:03.188ZINFOnexus (BackgroundTasks): received new VMM runtime state from sled agent background_task = instance_watcher file = nexus/src/app/instance.rs:2140 instance_id = 5f8711b0-0cf9-4ef2-aaaa-3cc9f32ebe3c migration_state = Migrations { migration_in: None, migration_out: None } propolis_id = b931750f-371f-4349-9ab8-e4868c2650e3 sled_id = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa vmm_id = b931750f-371f-4349-9ab8-e4868c2650e3 vmm_state = VmmRuntimeState { state: Running, gen: Generation(4), time_updated: 2025-08-26T21:57:01.346616291Z }
120192025-08-26T21:57:03.188ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher instance_id = 5f8711b0-0cf9-4ef2-aaaa-3cc9f32ebe3c roles = RoleSet { roles: {} } sled_id = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa vmm_id = b931750f-371f-4349-9ab8-e4868c2650e3
120202025-08-26T21:57:03.188ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher instance_id = 5f8711b0-0cf9-4ef2-aaaa-3cc9f32ebe3c resource = Database result = Ok(()) sled_id = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa vmm_id = b931750f-371f-4349-9ab8-e4868c2650e3
120212025-08-26T21:57:03.206ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:550 pruned_instances = 0 total_completed = 1 total_failed = 0 total_incomplete = 0 total_instances = 1 update_sagas_queued = 0
120222025-08-26T21:57:03.208ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 338.603543ms iteration = 23 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{"running":1},"pruned_instances":0,"total_instances":1,"update_sagas_queued":0}
120232025-08-26T21:57:03.223ZINFOnexus (BackgroundTasks): VPC route manager sled dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
120242025-08-26T21:57:03.223ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
120252025-08-26T21:57:03.223ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "279055e0-9b2f-489c-b8fa-ca673fd72e8f", "content-length": "583", "date": "Tue, 26 Aug 2025 21:57:02 GMT"} })
120262025-08-26T21:57:03.223ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
120272025-08-26T21:57:03.223ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
120282025-08-26T21:57:03.229ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
120292025-08-26T21:57:03.229ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
120302025-08-26T21:57:03.231ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 398.36553ms iteration = 23 status = null
120312025-08-26T21:57:03.237ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {a751b6c0-c32c-4656-871a-c6c5c7b72e38: {10.151.2.176: {31e4670d-43eb-47bb-9561-246959f0a73c}, 10.151.2.175: {31e4670d-43eb-47bb-9561-246959f0a73c}}} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
120322025-08-26T21:57:03.237ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
120332025-08-26T21:57:03.238ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "77ac777c-8291-44e2-8ea0-bb22cfdd0a85", "date": "Tue, 26 Aug 2025 21:57:02 GMT"} })
120342025-08-26T21:57:03.238ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
120352025-08-26T21:57:03.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
120362025-08-26T21:57:03.241ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
120372025-08-26T21:57:03.241ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
120382025-08-26T21:57:03.244ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
120392025-08-26T21:57:03.244ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
120402025-08-26T21:57:03.247ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
120412025-08-26T21:57:03.247ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
120422025-08-26T21:57:03.249ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
120432025-08-26T21:57:03.249ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
120442025-08-26T21:57:03.252ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
120452025-08-26T21:57:03.252ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
120462025-08-26T21:57:03.255ZINFOnexus (BackgroundTasks): VPC route manager sled dd0c655a-95c2-48fc-b4c3-9733ed4b0faa push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
120472025-08-26T21:57:03.255ZINFOnexus (BackgroundTasks): VPC route manager sled dd0c655a-95c2-48fc-b4c3-9733ed4b0faa push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
120482025-08-26T21:57:03.255ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
120492025-08-26T21:57:03.255ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "a330de63-e9e3-425c-a95d-8a99dd514555", "date": "Tue, 26 Aug 2025 21:57:02 GMT"} })
120502025-08-26T21:57:03.257ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 387.764351ms iteration = 23 status = {}
120512025-08-26T21:57:03.293ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
120522025-08-26T21:57:03.326ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 493.29411ms iteration = 23 status = {}
120532025-08-26T21:57:03.331ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
120542025-08-26T21:57:03.331ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "5ee4aa8c-1711-40a5-b482-e176913c3c21", "content-length": "810", "date": "Tue, 26 Aug 2025 21:57:03 GMT"} })
120552025-08-26T21:57:03.331ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {VirtualNetworkInterfaceHost { physical_host_ip: fd00:1122:3344:101::1, virtual_ip: 172.30.0.5, virtual_mac: MacAddr(MacAddr6([168, 64, 37, 240, 0, 0])), vni: Vni(8860997) }} sled = sock
120562025-08-26T21:57:03.331ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
120572025-08-26T21:57:03.331ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
120582025-08-26T21:57:03.332ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 462.981556ms iteration = 23 status = {}
120592025-08-26T21:57:03.389ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
120602025-08-26T21:57:03.390ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "93e7097f-7dd2-42dc-8130-758d278a9bd9", "date": "Tue, 26 Aug 2025 21:57:03 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
120612025-08-26T21:57:03.390ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
120622025-08-26T21:57:03.391ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "5d9dfe79-f7e9-4b96-a584-70ad369b7226", "date": "Tue, 26 Aug 2025 21:57:03 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
120632025-08-26T21:57:03.391ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
120642025-08-26T21:57:03.393ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "62749a8f-16da-4a44-b706-216e16b4a419", "date": "Tue, 26 Aug 2025 21:57:03 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
120652025-08-26T21:57:03.393ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
120662025-08-26T21:57:03.395ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
120672025-08-26T21:57:03.396ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 562.9385ms iteration = 14 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
120682025-08-26T21:57:03.408ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
120692025-08-26T21:57:03.409ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "e65cccc7-f3f6-4e52-ac68-9a647d507ca0", "date": "Tue, 26 Aug 2025 21:57:03 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
120702025-08-26T21:57:03.409ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
120712025-08-26T21:57:03.411ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "21b25168-a31b-4b13-be89-4686978c878a", "date": "Tue, 26 Aug 2025 21:57:03 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
120722025-08-26T21:57:03.411ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
120732025-08-26T21:57:03.413ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
120742025-08-26T21:57:03.413ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 580.172868ms iteration = 14 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
120752025-08-26T21:57:03.474ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
120762025-08-26T21:57:03.474ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
120772025-08-26T21:57:03.476ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: d24c5c55-3a21-4a91-aeb4-45362e822df6, rack_id: e37a46f2-f789-4ca4-9841-f8c6dd705964, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(88d84f37-a4c8-4abc-af1f-27987b0d39d9) } rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0
120782025-08-26T21:57:03.476ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
120792025-08-26T21:57:03.476ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
120802025-08-26T21:57:03.489ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1832 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
120812025-08-26T21:57:03.489ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1859 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 88d84f37-a4c8-4abc-af1f-27987b0d39d9
120822025-08-26T21:57:03.489ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
120832025-08-26T21:57:03.489ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
120842025-08-26T21:57:03.489ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
120852025-08-26T21:57:03.489ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
120862025-08-26T21:57:03.489ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
120872025-08-26T21:57:03.489ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2006 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
120882025-08-26T21:57:03.489ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:408 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
120892025-08-26T21:57:03.489ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2042 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
120902025-08-26T21:57:03.489ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
120912025-08-26T21:57:03.489ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
120922025-08-26T21:57:03.490ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
120932025-08-26T21:57:03.492ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0
120942025-08-26T21:57:03.492ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
120952025-08-26T21:57:03.577ZDEBGnexus (producer-registration-task): registering as oximeter producer with Nexus registration_count = 4
120962025-08-26T21:57:03.577ZDEBGnexus (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::c]:12221
120972025-08-26T21:57:03.608ZDEBGnexus (producer-registration-task): client request body = Some(Body) method = POST uri = http://[fd00:1122:3344:101::c]:12221/metrics/producers
120982025-08-26T21:57:03.608ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:1025 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::c]:37649
120992025-08-26T21:57:03.608ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "57a1a589-278e-490f-a8d0-ae323d2c5ca6", "date": "Tue, 26 Aug 2025 21:57:03 GMT"} })
121002025-08-26T21:57:03.608ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch = Switch0
121012025-08-26T21:57:03.608ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
121022025-08-26T21:57:03.608ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:37649 req_id = 18b51f9b-739e-4dd3-ad48-88dc2ab0d20b roles = RoleSet { roles: {} } uri = /metrics/producers
121032025-08-26T21:57:03.608ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "bfbeb9d0-e887-4dec-aa32-892e95eae46c", "content-length": "682", "date": "Tue, 26 Aug 2025 21:57:03 GMT"} })
121042025-08-26T21:57:03.608ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
121052025-08-26T21:57:03.608ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:37649 req_id = 18b51f9b-739e-4dd3-ad48-88dc2ab0d20b resource = Database result = Ok(()) uri = /metrics/producers
121062025-08-26T21:57:03.609ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
121072025-08-26T21:57:03.612ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
121082025-08-26T21:57:03.612ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
121092025-08-26T21:57:03.616ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
121102025-08-26T21:57:03.616ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
121112025-08-26T21:57:03.618ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
121122025-08-26T21:57:03.618ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
121132025-08-26T21:57:03.626ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 756.77928ms iteration = 23 status = {}
121142025-08-26T21:57:03.650ZINFOnexus: assigned collector to new producer collector_id = f843ff0e-eef8-49c4-b428-23e15df1def3 file = nexus/src/app/oximeter.rs:76 producer_id = 5d0959e5-47a6-4bf3-a52b-57950b9d900d
121152025-08-26T21:57:03.650ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 42076 local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:37649 req_id = 18b51f9b-739e-4dd3-ad48-88dc2ab0d20b response_code = 201 uri = /metrics/producers
121162025-08-26T21:57:03.650ZDEBGnexus (producer-registration-task): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::c]:12221/metrics/producers", status: 201, headers: {"content-type": "application/json", "x-request-id": "18b51f9b-739e-4dd3-ad48-88dc2ab0d20b", "content-length": "41", "date": "Tue, 26 Aug 2025 21:57:03 GMT"} })
121172025-08-26T21:57:03.652ZDEBGnexus (producer-registration-task): successfully registered with Nexus, pausing until time to renew lease lease_duration = 600s wait_period = 150s
121182025-08-26T21:57:05.355ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 3608 local_addr = [fd00:1122:3344:101::c]:42016 method = GET remote_addr = [fd00:1122:3344:101::d]:37550 req_id = a0d6ef90-52a2-4d4b-8641-96743544e047 response_code = 200 uri = /5d0959e5-47a6-4bf3-a52b-57950b9d900d
121192025-08-26T21:57:07.868ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 134 reason = Timeout
121202025-08-26T21:57:07.868ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
121212025-08-26T21:57:07.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
121222025-08-26T21:57:07.871ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
121232025-08-26T21:57:07.874ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
121242025-08-26T21:57:07.875ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
121252025-08-26T21:57:07.875ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
121262025-08-26T21:57:07.878ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
121272025-08-26T21:57:07.878ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 9.354761ms iteration = 134 status = {"chicken_switches_updated":false}
121282025-08-26T21:57:12.868ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 68 reason = Timeout
121292025-08-26T21:57:12.868ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
121302025-08-26T21:57:12.868ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 135 reason = Timeout
121312025-08-26T21:57:12.868ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
121322025-08-26T21:57:12.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
121332025-08-26T21:57:12.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
121342025-08-26T21:57:12.870ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
121352025-08-26T21:57:12.871ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 68 reason = Timeout
121362025-08-26T21:57:12.871ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
121372025-08-26T21:57:12.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
121382025-08-26T21:57:12.873ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
121392025-08-26T21:57:12.874ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
121402025-08-26T21:57:12.874ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
121412025-08-26T21:57:12.874ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
121422025-08-26T21:57:12.876ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
121432025-08-26T21:57:12.876ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.425498ms iteration = 135 status = {"chicken_switches_updated":false}
121442025-08-26T21:57:12.876ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
121452025-08-26T21:57:12.876ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
121462025-08-26T21:57:12.876ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
121472025-08-26T21:57:12.878ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
121482025-08-26T21:57:12.878ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
121492025-08-26T21:57:12.910ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
121502025-08-26T21:57:12.910ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
121512025-08-26T21:57:12.921ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
121522025-08-26T21:57:12.924ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
121532025-08-26T21:57:12.924ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
121542025-08-26T21:57:12.924ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
121552025-08-26T21:57:12.950ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
121562025-08-26T21:57:12.950ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 78.712654ms iteration = 68 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
121572025-08-26T21:57:12.959ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 91.065553ms iteration = 68 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
121582025-08-26T21:57:15.344ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:1025 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::d]:41982
121592025-08-26T21:57:15.344ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:41982 req_id = fd2ada60-24c0-4e4d-8e97-f02b6f95b24f roles = RoleSet { roles: {} } uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?sort_by=id_ascending
121602025-08-26T21:57:15.345ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:41982 req_id = fd2ada60-24c0-4e4d-8e97-f02b6f95b24f resource = Database result = Ok(()) uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?sort_by=id_ascending
121612025-08-26T21:57:15.347ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 2877 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:41982 req_id = fd2ada60-24c0-4e4d-8e97-f02b6f95b24f response_code = 200 uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?sort_by=id_ascending
121622025-08-26T21:57:15.347ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:41982 req_id = e9c05a81-95fb-41ed-91db-613de60e319b roles = RoleSet { roles: {} } uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMzhjYTg4YS1lZGE1LTQwZGEtOGE4MS1hN2UzMzdhMmM4ZmIifX0%3D
121632025-08-26T21:57:15.348ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:41982 req_id = e9c05a81-95fb-41ed-91db-613de60e319b resource = Database result = Ok(()) uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMzhjYTg4YS1lZGE1LTQwZGEtOGE4MS1hN2UzMzdhMmM4ZmIifX0%3D
121642025-08-26T21:57:15.350ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 2437 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:41982 req_id = e9c05a81-95fb-41ed-91db-613de60e319b response_code = 200 uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMzhjYTg4YS1lZGE1LTQwZGEtOGE4MS1hN2UzMzdhMmM4ZmIifX0%3D
121652025-08-26T21:57:15.351ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 363 local_addr = [fd00:1122:3344:101::c]:42016 method = GET remote_addr = [fd00:1122:3344:101::d]:37550 req_id = 3778f7f2-486d-4413-9a9a-051f89853267 response_code = 200 uri = /5d0959e5-47a6-4bf3-a52b-57950b9d900d
121662025-08-26T21:57:17.868ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 136 reason = Timeout
121672025-08-26T21:57:17.869ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
121682025-08-26T21:57:17.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
121692025-08-26T21:57:17.923ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
121702025-08-26T21:57:17.927ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
121712025-08-26T21:57:17.927ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
121722025-08-26T21:57:17.928ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
121732025-08-26T21:57:17.932ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
121742025-08-26T21:57:17.932ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 63.195385ms iteration = 136 status = {"chicken_switches_updated":false}
121752025-08-26T21:57:22.868ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 69 reason = Timeout
121762025-08-26T21:57:22.868ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
121772025-08-26T21:57:22.868ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 137 reason = Timeout
121782025-08-26T21:57:22.868ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
121792025-08-26T21:57:22.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
121802025-08-26T21:57:22.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
121812025-08-26T21:57:22.870ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
121822025-08-26T21:57:22.870ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
121832025-08-26T21:57:22.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
121842025-08-26T21:57:22.871ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 69 reason = Timeout
121852025-08-26T21:57:22.871ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
121862025-08-26T21:57:22.872ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
121872025-08-26T21:57:22.872ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.521693ms iteration = 69 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
121882025-08-26T21:57:22.873ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
121892025-08-26T21:57:22.873ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
121902025-08-26T21:57:22.873ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
121912025-08-26T21:57:22.874ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
121922025-08-26T21:57:22.875ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
121932025-08-26T21:57:22.875ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.430717ms iteration = 137 status = {"chicken_switches_updated":false}
121942025-08-26T21:57:22.876ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
121952025-08-26T21:57:22.876ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
121962025-08-26T21:57:22.877ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
121972025-08-26T21:57:22.879ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
121982025-08-26T21:57:22.879ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
121992025-08-26T21:57:22.880ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
122002025-08-26T21:57:22.883ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
122012025-08-26T21:57:22.883ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
122022025-08-26T21:57:22.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
122032025-08-26T21:57:22.895ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
122042025-08-26T21:57:22.895ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 24.103943ms iteration = 69 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
122052025-08-26T21:57:25.352ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 320 local_addr = [fd00:1122:3344:101::c]:42016 method = GET remote_addr = [fd00:1122:3344:101::d]:37550 req_id = 6795c91f-4f35-4f4c-a083-6223f9d23ecb response_code = 200 uri = /5d0959e5-47a6-4bf3-a52b-57950b9d900d
122062025-08-26T21:57:27.868ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 138 reason = Timeout
122072025-08-26T21:57:27.868ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
122082025-08-26T21:57:27.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
122092025-08-26T21:57:27.870ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
122102025-08-26T21:57:27.874ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
122112025-08-26T21:57:27.874ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
122122025-08-26T21:57:27.874ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
122132025-08-26T21:57:27.876ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
122142025-08-26T21:57:27.876ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.634363ms iteration = 138 status = {"chicken_switches_updated":false}
122152025-08-26T21:57:32.833ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 24 reason = Timeout
122162025-08-26T21:57:32.833ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 24 reason = Timeout
122172025-08-26T21:57:32.833ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 24 reason = Timeout
122182025-08-26T21:57:32.833ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
122192025-08-26T21:57:32.833ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
122202025-08-26T21:57:32.833ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
122212025-08-26T21:57:32.833ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
122222025-08-26T21:57:32.835ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1556, tv_nsec: 694918190 } })
122232025-08-26T21:57:32.836ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1205
122242025-08-26T21:57:32.836ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1556, tv_nsec: 695591015 } })
122252025-08-26T21:57:32.836ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.805725ms iteration = 24 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
122262025-08-26T21:57:32.836ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1205
122272025-08-26T21:57:32.868ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 24 reason = Timeout
122282025-08-26T21:57:32.868ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 12.53\u{b5}s iteration = 24 status = {"disabled":true,"errors":[],"sps":[]}
122292025-08-26T21:57:32.868ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 24 reason = Timeout
122302025-08-26T21:57:32.868ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 24 reason = Timeout
122312025-08-26T21:57:32.868ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
122322025-08-26T21:57:32.868ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 24 reason = Timeout
122332025-08-26T21:57:32.868ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
122342025-08-26T21:57:32.868ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
122352025-08-26T21:57:32.868ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 24 reason = Timeout
122362025-08-26T21:57:32.868ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 70 reason = Timeout
122372025-08-26T21:57:32.868ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
122382025-08-26T21:57:32.868ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
122392025-08-26T21:57:32.868ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 24 reason = Timeout
122402025-08-26T21:57:32.868ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
122412025-08-26T21:57:32.868ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 24 reason = Timeout
122422025-08-26T21:57:32.868ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
122432025-08-26T21:57:32.869ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 24 reason = Timeout
122442025-08-26T21:57:32.869ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
122452025-08-26T21:57:32.869ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 24 reason = Timeout
122462025-08-26T21:57:32.869ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 24 reason = Timeout
122472025-08-26T21:57:32.869ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
122482025-08-26T21:57:32.869ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
122492025-08-26T21:57:32.869ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
122502025-08-26T21:57:32.869ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 24 reason = Timeout
122512025-08-26T21:57:32.869ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
122522025-08-26T21:57:32.869ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 24 reason = Timeout
122532025-08-26T21:57:32.869ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
122542025-08-26T21:57:32.869ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 24 reason = Timeout
122552025-08-26T21:57:32.869ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
122562025-08-26T21:57:32.869ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
122572025-08-26T21:57:32.869ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 26 reason = Timeout
122582025-08-26T21:57:32.869ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
122592025-08-26T21:57:32.869ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 24 reason = Timeout
122602025-08-26T21:57:32.869ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
122612025-08-26T21:57:32.869ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 139 reason = Timeout
122622025-08-26T21:57:32.869ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
122632025-08-26T21:57:32.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
122642025-08-26T21:57:32.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
122652025-08-26T21:57:32.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
122662025-08-26T21:57:32.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
122672025-08-26T21:57:32.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
122682025-08-26T21:57:32.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
122692025-08-26T21:57:32.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
122702025-08-26T21:57:32.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
122712025-08-26T21:57:32.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
122722025-08-26T21:57:32.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
122732025-08-26T21:57:32.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
122742025-08-26T21:57:32.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
122752025-08-26T21:57:32.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
122762025-08-26T21:57:32.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
122772025-08-26T21:57:32.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
122782025-08-26T21:57:32.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
122792025-08-26T21:57:32.871ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
122802025-08-26T21:57:32.871ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
122812025-08-26T21:57:32.871ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
122822025-08-26T21:57:32.871ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 70 reason = Timeout
122832025-08-26T21:57:32.871ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
122842025-08-26T21:57:32.871ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
122852025-08-26T21:57:32.871ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 2.489076ms iteration = 24 status = {"errors":[],"garbage_collect_requested":[]}
122862025-08-26T21:57:32.871ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
122872025-08-26T21:57:32.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
122882025-08-26T21:57:32.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
122892025-08-26T21:57:32.873ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.329248ms iteration = 70 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
122902025-08-26T21:57:32.875ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
122912025-08-26T21:57:32.875ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
122922025-08-26T21:57:32.875ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
122932025-08-26T21:57:32.875ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
122942025-08-26T21:57:32.875ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
122952025-08-26T21:57:32.875ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
122962025-08-26T21:57:32.877ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
122972025-08-26T21:57:32.877ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
122982025-08-26T21:57:32.877ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
122992025-08-26T21:57:32.879ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
123002025-08-26T21:57:32.879ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
123012025-08-26T21:57:32.879ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
123022025-08-26T21:57:32.880ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
123032025-08-26T21:57:32.880ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
123042025-08-26T21:57:32.881ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
123052025-08-26T21:57:32.881ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 12.783348ms iteration = 139 status = {"chicken_switches_updated":false}
123062025-08-26T21:57:32.890ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
123072025-08-26T21:57:32.890ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
123082025-08-26T21:57:32.890ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
123092025-08-26T21:57:32.893ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
123102025-08-26T21:57:32.893ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
123112025-08-26T21:57:32.893ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
123122025-08-26T21:57:32.903ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 34.485158ms iteration = 26 status = {"physical_disks_added":0}
123132025-08-26T21:57:32.906ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1213 zone_address = fd00:1122:3344:101::2
123142025-08-26T21:57:32.906ZDEBGnexus (MgsClient): client request background_task = nat_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
123152025-08-26T21:57:32.907ZDEBGnexus (MgsClient): client response background_task = nat_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "77e7d01f-0424-4683-bba3-e1e12b58f46c", "content-length": "26", "date": "Tue, 26 Aug 2025 21:57:32 GMT"} })
123162025-08-26T21:57:32.907ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1216 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
123172025-08-26T21:57:32.909ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
123182025-08-26T21:57:32.910ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1213 zone_address = fd00:1122:3344:101::2
123192025-08-26T21:57:32.910ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
123202025-08-26T21:57:32.911ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "878d1b82-ada0-4962-9d74-6fcac252afe8", "content-length": "26", "date": "Tue, 26 Aug 2025 21:57:32 GMT"} })
123212025-08-26T21:57:32.911ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1216 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
123222025-08-26T21:57:32.912ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
123232025-08-26T21:57:32.919ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
123242025-08-26T21:57:32.919ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
123252025-08-26T21:57:32.921ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 52.420792ms iteration = 24 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
123262025-08-26T21:57:32.921ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
123272025-08-26T21:57:32.922ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
123282025-08-26T21:57:32.923ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
123292025-08-26T21:57:32.923ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
123302025-08-26T21:57:32.923ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
123312025-08-26T21:57:32.924ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
123322025-08-26T21:57:32.925ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
123332025-08-26T21:57:32.925ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
123342025-08-26T21:57:32.925ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
123352025-08-26T21:57:32.925ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
123362025-08-26T21:57:32.925ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
123372025-08-26T21:57:32.925ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
123382025-08-26T21:57:32.926ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1556, tv_nsec: 785656555 } })
123392025-08-26T21:57:32.927ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1205 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
123402025-08-26T21:57:32.928ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 59.580323ms iteration = 24 status = {"errors":[],"requests_created_ok":[]}
123412025-08-26T21:57:32.930ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
123422025-08-26T21:57:32.930ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
123432025-08-26T21:57:32.931ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
123442025-08-26T21:57:32.931ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
123452025-08-26T21:57:32.932ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
123462025-08-26T21:57:32.932ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
123472025-08-26T21:57:32.932ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
123482025-08-26T21:57:32.933ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
123492025-08-26T21:57:32.933ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
123502025-08-26T21:57:32.933ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
123512025-08-26T21:57:32.933ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
123522025-08-26T21:57:32.934ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
123532025-08-26T21:57:32.935ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
123542025-08-26T21:57:32.935ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
123552025-08-26T21:57:32.935ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
123562025-08-26T21:57:32.935ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
123572025-08-26T21:57:32.935ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
123582025-08-26T21:57:32.935ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
123592025-08-26T21:57:32.936ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 67.302237ms iteration = 24 status = {"errors":[],"finish_invoked_ok":[]}
123602025-08-26T21:57:32.937ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
123612025-08-26T21:57:32.937ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
123622025-08-26T21:57:32.937ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
123632025-08-26T21:57:32.937ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
123642025-08-26T21:57:32.937ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
123652025-08-26T21:57:32.938ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
123662025-08-26T21:57:32.938ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
123672025-08-26T21:57:32.938ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
123682025-08-26T21:57:32.938ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
123692025-08-26T21:57:32.940ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
123702025-08-26T21:57:32.940ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: dd0c655a-95c2-48fc-b4c3-9733ed4b0faa, lookup_type: ById(dd0c655a-95c2-48fc-b4c3-9733ed4b0faa) } result = Ok(())
123712025-08-26T21:57:32.940ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
123722025-08-26T21:57:32.941ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
123732025-08-26T21:57:32.942ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
123742025-08-26T21:57:32.942ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
123752025-08-26T21:57:32.942ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
123762025-08-26T21:57:32.943ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 74.114711ms iteration = 24 status = 0
123772025-08-26T21:57:32.964ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 95.329159ms iteration = 24 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
123782025-08-26T21:57:32.970ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
123792025-08-26T21:57:32.970ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
123802025-08-26T21:57:32.970ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
123812025-08-26T21:57:32.971ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
123822025-08-26T21:57:32.978ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 109.806155ms iteration = 24 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
123832025-08-26T21:57:32.982ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
123842025-08-26T21:57:32.983ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:550 pruned_instances = 1 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
123852025-08-26T21:57:32.983ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 114.781577ms iteration = 24 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":1,"total_instances":0,"update_sagas_queued":0}
123862025-08-26T21:57:32.985ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
123872025-08-26T21:57:32.985ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
123882025-08-26T21:57:32.986ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
123892025-08-26T21:57:32.989ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:407
123902025-08-26T21:57:32.989ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 120.22215ms iteration = 24 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
123912025-08-26T21:57:33.003ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
123922025-08-26T21:57:33.003ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 132.130738ms iteration = 70 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
123932025-08-26T21:57:33.006ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
123942025-08-26T21:57:33.006ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
123952025-08-26T21:57:33.006ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 137.174133ms iteration = 24 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
123962025-08-26T21:57:33.006ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
123972025-08-26T21:57:33.006ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
123982025-08-26T21:57:33.008ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 175.736001ms iteration = 24 status = null
123992025-08-26T21:57:33.023ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
124002025-08-26T21:57:33.023ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 190.416693ms iteration = 24 status = {}
124012025-08-26T21:57:33.023ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
124022025-08-26T21:57:33.030ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1213 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 zone_address = fd00:1122:3344:101::2
124032025-08-26T21:57:33.030ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
124042025-08-26T21:57:33.030ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "d2c66aa1-a1c9-4cb3-9147-5653d0bcb700", "content-length": "26", "date": "Tue, 26 Aug 2025 21:57:32 GMT"} })
124052025-08-26T21:57:33.030ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1216 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
124062025-08-26T21:57:33.032ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
124072025-08-26T21:57:33.042ZINFOnexus (BackgroundTasks): VPC route manager sled dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
124082025-08-26T21:57:33.042ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
124092025-08-26T21:57:33.042ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "91fe9edd-9040-4d63-ad82-2f7b9516eeb3", "content-length": "372", "date": "Tue, 26 Aug 2025 21:57:33 GMT"} })
124102025-08-26T21:57:33.042ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
124112025-08-26T21:57:33.043ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
124122025-08-26T21:57:33.057ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
124132025-08-26T21:57:33.057ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
124142025-08-26T21:57:33.057ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "bad8189a-6ba4-4d4b-9307-d9e2e9b6021b", "date": "Tue, 26 Aug 2025 21:57:33 GMT"} })
124152025-08-26T21:57:33.057ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
124162025-08-26T21:57:33.057ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
124172025-08-26T21:57:33.059ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
124182025-08-26T21:57:33.060ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
124192025-08-26T21:57:33.062ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
124202025-08-26T21:57:33.062ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
124212025-08-26T21:57:33.065ZINFOnexus (BackgroundTasks): VPC route manager sled dd0c655a-95c2-48fc-b4c3-9733ed4b0faa push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
124222025-08-26T21:57:33.065ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
124232025-08-26T21:57:33.066ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "a677a7a0-9084-4d19-8e31-99f77e989281", "date": "Tue, 26 Aug 2025 21:57:33 GMT"} })
124242025-08-26T21:57:33.068ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 199.1761ms iteration = 24 status = {}
124252025-08-26T21:57:33.134ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
124262025-08-26T21:57:33.135ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
124272025-08-26T21:57:33.137ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: d24c5c55-3a21-4a91-aeb4-45362e822df6, rack_id: e37a46f2-f789-4ca4-9841-f8c6dd705964, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(88d84f37-a4c8-4abc-af1f-27987b0d39d9) } rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0
124282025-08-26T21:57:33.137ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
124292025-08-26T21:57:33.137ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
124302025-08-26T21:57:33.141ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
124312025-08-26T21:57:33.142ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
124322025-08-26T21:57:33.143ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
124332025-08-26T21:57:33.145ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
124342025-08-26T21:57:33.145ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
124352025-08-26T21:57:33.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
124362025-08-26T21:57:33.147ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
124372025-08-26T21:57:33.149ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
124382025-08-26T21:57:33.149ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
124392025-08-26T21:57:33.149ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
124402025-08-26T21:57:33.149ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1832 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
124412025-08-26T21:57:33.149ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1859 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 88d84f37-a4c8-4abc-af1f-27987b0d39d9
124422025-08-26T21:57:33.150ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
124432025-08-26T21:57:33.150ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
124442025-08-26T21:57:33.150ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
124452025-08-26T21:57:33.150ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
124462025-08-26T21:57:33.150ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
124472025-08-26T21:57:33.150ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2006 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
124482025-08-26T21:57:33.150ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:408 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
124492025-08-26T21:57:33.150ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2042 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
124502025-08-26T21:57:33.150ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
124512025-08-26T21:57:33.150ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
124522025-08-26T21:57:33.150ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
124532025-08-26T21:57:33.152ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0
124542025-08-26T21:57:33.152ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
124552025-08-26T21:57:33.257ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
124562025-08-26T21:57:33.258ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "f110ae73-0015-4798-a6b0-c4a2f0010e60", "content-length": "691", "date": "Tue, 26 Aug 2025 21:57:33 GMT"} })
124572025-08-26T21:57:33.258ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
124582025-08-26T21:57:33.258ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
124592025-08-26T21:57:33.258ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
124602025-08-26T21:57:33.259ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 390.491793ms iteration = 24 status = {}
124612025-08-26T21:57:33.264ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "13916d44-c1f2-49dc-b940-b3eb5b84059f", "date": "Tue, 26 Aug 2025 21:57:33 GMT"} })
124622025-08-26T21:57:33.264ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch = Switch0
124632025-08-26T21:57:33.264ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
124642025-08-26T21:57:33.264ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "3f35ebd6-1f27-4035-b17e-14610ca10f54", "content-length": "682", "date": "Tue, 26 Aug 2025 21:57:33 GMT"} })
124652025-08-26T21:57:33.264ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
124662025-08-26T21:57:33.265ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
124672025-08-26T21:57:33.267ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
124682025-08-26T21:57:33.268ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
124692025-08-26T21:57:33.271ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
124702025-08-26T21:57:33.271ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
124712025-08-26T21:57:33.273ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
124722025-08-26T21:57:33.273ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
124732025-08-26T21:57:33.281ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 412.469928ms iteration = 24 status = {}
124742025-08-26T21:57:35.352ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 288 local_addr = [fd00:1122:3344:101::c]:42016 method = GET remote_addr = [fd00:1122:3344:101::d]:37550 req_id = 3746e7d4-5830-420d-9ec7-9cc601ae52dd response_code = 200 uri = /5d0959e5-47a6-4bf3-a52b-57950b9d900d
124752025-08-26T21:57:37.868ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 140 reason = Timeout
124762025-08-26T21:57:37.868ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
124772025-08-26T21:57:37.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
124782025-08-26T21:57:37.870ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
124792025-08-26T21:57:37.873ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
124802025-08-26T21:57:37.873ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
124812025-08-26T21:57:37.873ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
124822025-08-26T21:57:37.876ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
124832025-08-26T21:57:37.876ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.949419ms iteration = 140 status = {"chicken_switches_updated":false}
124842025-08-26T21:57:42.868ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 71 reason = Timeout
124852025-08-26T21:57:42.868ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
124862025-08-26T21:57:42.868ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 141 reason = Timeout
124872025-08-26T21:57:42.868ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
124882025-08-26T21:57:42.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
124892025-08-26T21:57:42.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
124902025-08-26T21:57:42.871ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 71 reason = Timeout
124912025-08-26T21:57:42.871ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
124922025-08-26T21:57:42.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
124932025-08-26T21:57:42.873ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
124942025-08-26T21:57:42.876ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
124952025-08-26T21:57:42.876ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
124962025-08-26T21:57:42.877ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
124972025-08-26T21:57:42.879ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
124982025-08-26T21:57:42.879ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
124992025-08-26T21:57:42.881ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
125002025-08-26T21:57:42.883ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
125012025-08-26T21:57:42.883ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
125022025-08-26T21:57:42.884ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
125032025-08-26T21:57:42.900ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
125042025-08-26T21:57:42.900ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 29.086675ms iteration = 71 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
125052025-08-26T21:57:42.904ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
125062025-08-26T21:57:42.905ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
125072025-08-26T21:57:42.910ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
125082025-08-26T21:57:42.913ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
125092025-08-26T21:57:42.913ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
125102025-08-26T21:57:42.913ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
125112025-08-26T21:57:42.915ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
125122025-08-26T21:57:42.915ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 46.549749ms iteration = 141 status = {"chicken_switches_updated":false}
125132025-08-26T21:57:42.941ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 72.874983ms iteration = 71 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
125142025-08-26T21:57:45.344ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:41982 req_id = 0a6fd8be-21ff-4b5f-978e-719df720aca4 roles = RoleSet { roles: {} } uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?sort_by=id_ascending
125152025-08-26T21:57:45.344ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:41982 req_id = 0a6fd8be-21ff-4b5f-978e-719df720aca4 resource = Database result = Ok(()) uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?sort_by=id_ascending
125162025-08-26T21:57:45.347ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 3427 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:41982 req_id = 0a6fd8be-21ff-4b5f-978e-719df720aca4 response_code = 200 uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?sort_by=id_ascending
125172025-08-26T21:57:45.348ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:41982 req_id = 5475d37c-af41-41f1-83a4-39cde6d6fae7 roles = RoleSet { roles: {} } uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMzhjYTg4YS1lZGE1LTQwZGEtOGE4MS1hN2UzMzdhMmM4ZmIifX0%3D
125182025-08-26T21:57:45.348ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:41982 req_id = 5475d37c-af41-41f1-83a4-39cde6d6fae7 resource = Database result = Ok(()) uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMzhjYTg4YS1lZGE1LTQwZGEtOGE4MS1hN2UzMzdhMmM4ZmIifX0%3D
125192025-08-26T21:57:45.351ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 2800 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:41982 req_id = 5475d37c-af41-41f1-83a4-39cde6d6fae7 response_code = 200 uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMzhjYTg4YS1lZGE1LTQwZGEtOGE4MS1hN2UzMzdhMmM4ZmIifX0%3D
125202025-08-26T21:57:45.351ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 234 local_addr = [fd00:1122:3344:101::c]:42016 method = GET remote_addr = [fd00:1122:3344:101::d]:37550 req_id = 57bd2a9a-2128-41ca-878b-30ad374196db response_code = 200 uri = /5d0959e5-47a6-4bf3-a52b-57950b9d900d
125212025-08-26T21:57:47.868ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 142 reason = Timeout
125222025-08-26T21:57:47.868ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
125232025-08-26T21:57:47.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
125242025-08-26T21:57:47.871ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
125252025-08-26T21:57:47.874ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
125262025-08-26T21:57:47.874ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
125272025-08-26T21:57:47.874ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
125282025-08-26T21:57:47.876ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
125292025-08-26T21:57:47.876ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 8.273167ms iteration = 142 status = {"chicken_switches_updated":false}
125302025-08-26T21:57:52.868ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 72 reason = Timeout
125312025-08-26T21:57:52.868ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 143 reason = Timeout
125322025-08-26T21:57:52.868ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
125332025-08-26T21:57:52.868ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
125342025-08-26T21:57:52.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
125352025-08-26T21:57:52.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
125362025-08-26T21:57:52.870ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
125372025-08-26T21:57:52.871ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
125382025-08-26T21:57:52.871ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 72 reason = Timeout
125392025-08-26T21:57:52.871ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
125402025-08-26T21:57:52.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
125412025-08-26T21:57:52.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
125422025-08-26T21:57:52.873ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
125432025-08-26T21:57:52.874ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
125442025-08-26T21:57:52.874ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
125452025-08-26T21:57:52.874ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
125462025-08-26T21:57:52.876ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
125472025-08-26T21:57:52.876ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
125482025-08-26T21:57:52.876ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
125492025-08-26T21:57:52.876ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.745525ms iteration = 143 status = {"chicken_switches_updated":false}
125502025-08-26T21:57:52.876ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
125512025-08-26T21:57:52.878ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
125522025-08-26T21:57:52.878ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
125532025-08-26T21:57:52.880ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
125542025-08-26T21:57:52.882ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
125552025-08-26T21:57:52.882ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
125562025-08-26T21:57:52.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
125572025-08-26T21:57:52.899ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
125582025-08-26T21:57:52.899ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 27.964791ms iteration = 72 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
125592025-08-26T21:57:52.907ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 38.758503ms iteration = 72 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
125602025-08-26T21:57:55.352ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 284 local_addr = [fd00:1122:3344:101::c]:42016 method = GET remote_addr = [fd00:1122:3344:101::d]:37550 req_id = dd1d8aaf-d4ae-43fb-973a-fa4be6749f77 response_code = 200 uri = /5d0959e5-47a6-4bf3-a52b-57950b9d900d
125612025-08-26T21:57:57.868ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 144 reason = Timeout
125622025-08-26T21:57:57.868ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
125632025-08-26T21:57:57.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
125642025-08-26T21:57:57.870ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
125652025-08-26T21:57:57.873ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
125662025-08-26T21:57:57.873ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
125672025-08-26T21:57:57.873ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
125682025-08-26T21:57:57.875ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
125692025-08-26T21:57:57.875ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.325595ms iteration = 144 status = {"chicken_switches_updated":false}
125702025-08-26T21:58:02.832ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 13 reason = Timeout
125712025-08-26T21:58:02.832ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 13 reason = Timeout
125722025-08-26T21:58:02.832ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 25 reason = Timeout
125732025-08-26T21:58:02.832ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
125742025-08-26T21:58:02.832ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 15 reason = Timeout
125752025-08-26T21:58:02.832ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 25 reason = Timeout
125762025-08-26T21:58:02.832ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
125772025-08-26T21:58:02.832ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 15 reason = Timeout
125782025-08-26T21:58:02.832ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 25 reason = Timeout
125792025-08-26T21:58:02.832ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
125802025-08-26T21:58:02.832ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 13 reason = Timeout
125812025-08-26T21:58:02.832ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-08-26 21:48:02.832541361 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
125822025-08-26T21:58:02.832ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
125832025-08-26T21:58:02.832ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 13 reason = Timeout
125842025-08-26T21:58:02.832ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
125852025-08-26T21:58:02.832ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
125862025-08-26T21:58:02.832ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 13 reason = Timeout
125872025-08-26T21:58:02.832ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 13 reason = Timeout
125882025-08-26T21:58:02.832ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
125892025-08-26T21:58:02.832ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
125902025-08-26T21:58:02.832ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
125912025-08-26T21:58:02.832ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
125922025-08-26T21:58:02.832ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
125932025-08-26T21:58:02.833ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
125942025-08-26T21:58:02.833ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
125952025-08-26T21:58:02.833ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("852fd1a3-de9a-47e8-a667-5e8dab880ad3.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("c68f1501-294c-4874-92ca-8fc0546a5870.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("d741a72f-8745-4723-9e80-e1e72987266b.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("d741a72f-8745-4723-9e80-e1e72987266b.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:3::1)) }], valid_until: Instant { tv_sec: 1586, tv_nsec: 692203209 } })
125962025-08-26T21:58:02.833ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
125972025-08-26T21:58:02.833ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("60abf731-31e1-4111-a3e2-65da89eaed87.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("84c849eb-0949-41d9-ad02-9a5509144564.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("84c849eb-0949-41d9-ad02-9a5509144564.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::8)) }], valid_until: Instant { tv_sec: 1586, tv_nsec: 692253050 } })
125982025-08-26T21:58:02.833ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
125992025-08-26T21:58:02.833ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.178836ms iteration = 13 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
126002025-08-26T21:58:02.833ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } dns_group = internal
126012025-08-26T21:58:02.833ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
126022025-08-26T21:58:02.833ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.122585ms iteration = 13 status = {"addresses":["[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:3::1]:5353"]}
126032025-08-26T21:58:02.833ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
126042025-08-26T21:58:02.833ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
126052025-08-26T21:58:02.834ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
126062025-08-26T21:58:02.834ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1586, tv_nsec: 693925388 } })
126072025-08-26T21:58:02.834ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
126082025-08-26T21:58:02.834ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1205
126092025-08-26T21:58:02.835ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 14 reason = Timeout
126102025-08-26T21:58:02.835ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-08-26 21:48:02.832541361 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
126112025-08-26T21:58:02.835ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:91 reason = blueprint disabled target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8
126122025-08-26T21:58:02.835ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 13.631\u{b5}s iteration = 14 status = {"enabled":false,"target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8"}
126132025-08-26T21:58:02.835ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 2.680631ms iteration = 13 status = {"expiration":"2025-08-26T21:48:02.832541361Z","pruned":[]}
126142025-08-26T21:58:02.835ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
126152025-08-26T21:58:02.835ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1586, tv_nsec: 695137605 } })
126162025-08-26T21:58:02.835ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.576471ms iteration = 25 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
126172025-08-26T21:58:02.836ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
126182025-08-26T21:58:02.836ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1205
126192025-08-26T21:58:02.837ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
126202025-08-26T21:58:02.837ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
126212025-08-26T21:58:02.837ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
126222025-08-26T21:58:02.839ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
126232025-08-26T21:58:02.839ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
126242025-08-26T21:58:02.839ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
126252025-08-26T21:58:02.839ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
126262025-08-26T21:58:02.841ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
126272025-08-26T21:58:02.844ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
126282025-08-26T21:58:02.844ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
126292025-08-26T21:58:02.844ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
126302025-08-26T21:58:02.846ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
126312025-08-26T21:58:02.846ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
126322025-08-26T21:58:02.847ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
126332025-08-26T21:58:02.849ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
126342025-08-26T21:58:02.849ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
126352025-08-26T21:58:02.850ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
126362025-08-26T21:58:02.859ZDEBGnexus (BackgroundTasks): activating background_task = crdb_node_id_collector iteration = 6 reason = Timeout
126372025-08-26T21:58:02.859ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 13 reason = Timeout
126382025-08-26T21:58:02.859ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
126392025-08-26T21:58:02.859ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 13 reason = Timeout
126402025-08-26T21:58:02.859ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
126412025-08-26T21:58:02.859ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:53
126422025-08-26T21:58:02.859ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
126432025-08-26T21:58:02.859ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 13 reason = Timeout
126442025-08-26T21:58:02.859ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
126452025-08-26T21:58:02.859ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 16 reason = Timeout
126462025-08-26T21:58:02.859ZDEBGnexus (BackgroundTasks): blueprint planning disabled, doing nothing background_task = blueprint_planner
126472025-08-26T21:58:02.859ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 8.12\u{b5}s iteration = 16 status = "Disabled"
126482025-08-26T21:58:02.859ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
126492025-08-26T21:58:02.859ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
126502025-08-26T21:58:02.859ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
126512025-08-26T21:58:02.859ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
126522025-08-26T21:58:02.859ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
126532025-08-26T21:58:02.860ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
126542025-08-26T21:58:02.860ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
126552025-08-26T21:58:02.860ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 13 reason = Timeout
126562025-08-26T21:58:02.860ZDEBGnexus (BackgroundTasks): activating background_task = webhook_deliverator iteration = 13 reason = Timeout
126572025-08-26T21:58:02.860ZDEBGnexus (BackgroundTasks): activating background_task = alert_dispatcher iteration = 13 reason = Timeout
126582025-08-26T21:58:02.860ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
126592025-08-26T21:58:02.860ZDEBGnexus (BackgroundTasks): roles background_task = webhook_deliverator roles = RoleSet { roles: {} }
126602025-08-26T21:58:02.860ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
126612025-08-26T21:58:02.860ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
126622025-08-26T21:58:02.860ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
126632025-08-26T21:58:02.860ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
126642025-08-26T21:58:02.860ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
126652025-08-26T21:58:02.860ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = webhook_deliverator resource = Database result = Ok(())
126662025-08-26T21:58:02.860ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
126672025-08-26T21:58:02.860ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = alert_dispatcher resource = Database result = Ok(())
126682025-08-26T21:58:02.860ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
126692025-08-26T21:58:02.861ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
126702025-08-26T21:58:02.862ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
126712025-08-26T21:58:02.862ZINFOnexus (BackgroundTasks): all webhook delivery tasks completed background_task = webhook_deliverator file = nexus/src/app/background/tasks/webhook_deliverator.rs:204 num_receivers = 0
126722025-08-26T21:58:02.862ZDEBGnexus (BackgroundTasks): activation complete background_task = webhook_deliverator elapsed = 2.63971ms iteration = 13 status = {"by_rx":{},"error":null}
126732025-08-26T21:58:02.863ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 3.09403ms iteration = 13 status = {"errors":[],"found_port_ok":[]}
126742025-08-26T21:58:02.863ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
126752025-08-26T21:58:02.863ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
126762025-08-26T21:58:02.863ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
126772025-08-26T21:58:02.863ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
126782025-08-26T21:58:02.863ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:170 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
126792025-08-26T21:58:02.863ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 4.235646ms iteration = 13 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
126802025-08-26T21:58:02.864ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
126812025-08-26T21:58:02.864ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = alert_dispatcher resource = Database result = Ok(())
126822025-08-26T21:58:02.864ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
126832025-08-26T21:58:02.864ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
126842025-08-26T21:58:02.865ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
126852025-08-26T21:58:02.865ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
126862025-08-26T21:58:02.866ZDEBGnexus (BackgroundTasks): activation complete background_task = alert_dispatcher elapsed = 5.844792ms iteration = 13 status = {"dispatched":[],"errors":[],"glob_version":"183.0.0","globs_reprocessed":{},"no_receivers":[]}
126872025-08-26T21:58:02.867ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
126882025-08-26T21:58:02.867ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
126892025-08-26T21:58:02.867ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
126902025-08-26T21:58:02.867ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
126912025-08-26T21:58:02.868ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 25 reason = Timeout
126922025-08-26T21:58:02.868ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 25 reason = Timeout
126932025-08-26T21:58:02.868ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 12.85\u{b5}s iteration = 25 status = {"disabled":true,"errors":[],"sps":[]}
126942025-08-26T21:58:02.868ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
126952025-08-26T21:58:02.868ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 25 reason = Timeout
126962025-08-26T21:58:02.868ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 25 reason = Timeout
126972025-08-26T21:58:02.868ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
126982025-08-26T21:58:02.868ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 27 reason = Timeout
126992025-08-26T21:58:02.868ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
127002025-08-26T21:58:02.868ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
127012025-08-26T21:58:02.868ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 25 reason = Timeout
127022025-08-26T21:58:02.868ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
127032025-08-26T21:58:02.868ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 25 reason = Timeout
127042025-08-26T21:58:02.868ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
127052025-08-26T21:58:02.868ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 73 reason = Timeout
127062025-08-26T21:58:02.868ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
127072025-08-26T21:58:02.868ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 145 reason = Timeout
127082025-08-26T21:58:02.868ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
127092025-08-26T21:58:02.868ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 25 reason = Timeout
127102025-08-26T21:58:02.868ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
127112025-08-26T21:58:02.868ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 25 reason = Timeout
127122025-08-26T21:58:02.868ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
127132025-08-26T21:58:02.868ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 25 reason = Timeout
127142025-08-26T21:58:02.868ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
127152025-08-26T21:58:02.868ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
127162025-08-26T21:58:02.868ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 25 reason = Timeout
127172025-08-26T21:58:02.868ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
127182025-08-26T21:58:02.868ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 25 reason = Timeout
127192025-08-26T21:58:02.868ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
127202025-08-26T21:58:02.868ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
127212025-08-26T21:58:02.868ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 25 reason = Timeout
127222025-08-26T21:58:02.868ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
127232025-08-26T21:58:02.868ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 25 reason = Timeout
127242025-08-26T21:58:02.868ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
127252025-08-26T21:58:02.868ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 25 reason = Timeout
127262025-08-26T21:58:02.868ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
127272025-08-26T21:58:02.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
127282025-08-26T21:58:02.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
127292025-08-26T21:58:02.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
127302025-08-26T21:58:02.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
127312025-08-26T21:58:02.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
127322025-08-26T21:58:02.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
127332025-08-26T21:58:02.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
127342025-08-26T21:58:02.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
127352025-08-26T21:58:02.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
127362025-08-26T21:58:02.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
127372025-08-26T21:58:02.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
127382025-08-26T21:58:02.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
127392025-08-26T21:58:02.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
127402025-08-26T21:58:02.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
127412025-08-26T21:58:02.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
127422025-08-26T21:58:02.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
127432025-08-26T21:58:02.869ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
127442025-08-26T21:58:02.869ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
127452025-08-26T21:58:02.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
127462025-08-26T21:58:02.869ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
127472025-08-26T21:58:02.870ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
127482025-08-26T21:58:02.870ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
127492025-08-26T21:58:02.872ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 73 reason = Timeout
127502025-08-26T21:58:02.872ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
127512025-08-26T21:58:02.872ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
127522025-08-26T21:58:02.872ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
127532025-08-26T21:58:02.873ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
127542025-08-26T21:58:02.873ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
127552025-08-26T21:58:02.873ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
127562025-08-26T21:58:02.873ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
127572025-08-26T21:58:02.874ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 5.207667ms iteration = 25 status = {"errors":[],"finish_invoked_ok":[]}
127582025-08-26T21:58:02.874ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
127592025-08-26T21:58:02.874ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
127602025-08-26T21:58:02.874ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
127612025-08-26T21:58:02.874ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
127622025-08-26T21:58:02.875ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
127632025-08-26T21:58:02.875ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
127642025-08-26T21:58:02.876ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
127652025-08-26T21:58:02.876ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
127662025-08-26T21:58:02.876ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
127672025-08-26T21:58:02.876ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
127682025-08-26T21:58:02.877ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
127692025-08-26T21:58:02.878ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 10.237231ms iteration = 73 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
127702025-08-26T21:58:02.879ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
127712025-08-26T21:58:02.879ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
127722025-08-26T21:58:02.879ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
127732025-08-26T21:58:02.879ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
127742025-08-26T21:58:02.879ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
127752025-08-26T21:58:02.879ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
127762025-08-26T21:58:02.879ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
127772025-08-26T21:58:02.879ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
127782025-08-26T21:58:02.879ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
127792025-08-26T21:58:02.879ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
127802025-08-26T21:58:02.879ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
127812025-08-26T21:58:02.880ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
127822025-08-26T21:58:02.880ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
127832025-08-26T21:58:02.880ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
127842025-08-26T21:58:02.880ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
127852025-08-26T21:58:02.880ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
127862025-08-26T21:58:02.880ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
127872025-08-26T21:58:02.881ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
127882025-08-26T21:58:02.881ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
127892025-08-26T21:58:02.881ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
127902025-08-26T21:58:02.881ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
127912025-08-26T21:58:02.881ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
127922025-08-26T21:58:02.881ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
127932025-08-26T21:58:02.881ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
127942025-08-26T21:58:02.881ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
127952025-08-26T21:58:02.881ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
127962025-08-26T21:58:02.881ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
127972025-08-26T21:58:02.881ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
127982025-08-26T21:58:02.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
127992025-08-26T21:58:02.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
128002025-08-26T21:58:02.882ZDEBGnexus (BackgroundTasks): activation complete background_task = crdb_node_id_collector elapsed = 23.340156ms iteration = 6 status = {"nsuccess":5}
128012025-08-26T21:58:02.883ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
128022025-08-26T21:58:02.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
128032025-08-26T21:58:02.883ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
128042025-08-26T21:58:02.883ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
128052025-08-26T21:58:02.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
128062025-08-26T21:58:02.885ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
128072025-08-26T21:58:02.885ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
128082025-08-26T21:58:02.885ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
128092025-08-26T21:58:02.885ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
128102025-08-26T21:58:02.885ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
128112025-08-26T21:58:02.885ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
128122025-08-26T21:58:02.885ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
128132025-08-26T21:58:02.886ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
128142025-08-26T21:58:02.886ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
128152025-08-26T21:58:02.887ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
128162025-08-26T21:58:02.887ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
128172025-08-26T21:58:02.887ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
128182025-08-26T21:58:02.888ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
128192025-08-26T21:58:02.889ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
128202025-08-26T21:58:02.889ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
128212025-08-26T21:58:02.889ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
128222025-08-26T21:58:02.889ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:550 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
128232025-08-26T21:58:02.889ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 20.744038ms iteration = 25 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
128242025-08-26T21:58:02.891ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
128252025-08-26T21:58:02.891ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 8e720f02-dde8-4423-a6e3-91698373805a dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
128262025-08-26T21:58:02.891ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
128272025-08-26T21:58:02.891ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
128282025-08-26T21:58:02.903ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
128292025-08-26T21:58:02.908ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-08-26T21:45:59.866217Z, time_modified: 2025-08-26T21:45:59.866217Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 9396aa55-de6f-4d6f-96db-d28e1c76fa9c, db_silo: Silo { identity: SiloIdentity { id: 9396aa55-de6f-4d6f-96db-d28e1c76fa9c, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-08-26T21:46:05.245521Z, time_modified: 2025-08-26T21:46:05.245521Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "e8529d720e4b0de099753dcdb373abb870af04baacd7c7dd850f5254858081af" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 9396aa55-de6f-4d6f-96db-d28e1c76fa9c, db_silo: Silo { identity: SiloIdentity { id: 9396aa55-de6f-4d6f-96db-d28e1c76fa9c, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-08-26T21:46:05.245521Z, time_modified: 2025-08-26T21:46:05.245521Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "e8529d720e4b0de099753dcdb373abb870af04baacd7c7dd850f5254858081af" }] }) }
128302025-08-26T21:58:02.908ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
128312025-08-26T21:58:02.908ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
128322025-08-26T21:58:02.908ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 76.572976ms iteration = 13 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"9396aa55-de6f-4d6f-96db-d28e1c76fa9c","tls_certs":["e8529d720e4b0de099753dcdb373abb870af04baacd7c7dd850f5254858081af"]}},"default_endpoint":{"silo_id":"9396aa55-de6f-4d6f-96db-d28e1c76fa9c","tls_certs":["e8529d720e4b0de099753dcdb373abb870af04baacd7c7dd850f5254858081af"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
128332025-08-26T21:58:02.909ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
128342025-08-26T21:58:02.909ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
128352025-08-26T21:58:02.909ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:138 found = 0 not_ready_to_be_deleted = 0
128362025-08-26T21:58:02.909ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 50.190632ms iteration = 13 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
128372025-08-26T21:58:02.913ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 45.065216ms iteration = 25 status = {"errors":[],"garbage_collect_requested":[]}
128382025-08-26T21:58:02.914ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
128392025-08-26T21:58:02.914ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 42.5951ms iteration = 73 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
128402025-08-26T21:58:02.917ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 8e720f02-dde8-4423-a6e3-91698373805a dns_zone_name = control-plane.oxide.internal found_so_far = 59 version = 1
128412025-08-26T21:58:02.917ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
128422025-08-26T21:58:02.917ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2025-08-26 21:46:04.065217 UTC dns_group = internal generation = 1
128432025-08-26T21:58:02.917ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 84.73397ms iteration = 13 status = {"generation":1}
128442025-08-26T21:58:02.930ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
128452025-08-26T21:58:02.930ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
128462025-08-26T21:58:02.930ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
128472025-08-26T21:58:02.930ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
128482025-08-26T21:58:02.930ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
128492025-08-26T21:58:02.932ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
128502025-08-26T21:58:02.932ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
128512025-08-26T21:58:02.932ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 64.205737ms iteration = 25 status = {"errors":[],"requests_created_ok":[]}
128522025-08-26T21:58:02.933ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
128532025-08-26T21:58:02.933ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
128542025-08-26T21:58:02.933ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
128552025-08-26T21:58:02.934ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
128562025-08-26T21:58:02.934ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
128572025-08-26T21:58:02.934ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
128582025-08-26T21:58:02.934ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 65.810633ms iteration = 25 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
128592025-08-26T21:58:02.934ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
128602025-08-26T21:58:02.936ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
128612025-08-26T21:58:02.936ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
128622025-08-26T21:58:02.936ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 67.634524ms iteration = 145 status = {"chicken_switches_updated":false}
128632025-08-26T21:58:02.936ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
128642025-08-26T21:58:02.937ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
128652025-08-26T21:58:02.938ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
128662025-08-26T21:58:02.938ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
128672025-08-26T21:58:02.938ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
128682025-08-26T21:58:02.939ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
128692025-08-26T21:58:02.939ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
128702025-08-26T21:58:02.940ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 71.569403ms iteration = 25 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
128712025-08-26T21:58:02.940ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
128722025-08-26T21:58:02.941ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
128732025-08-26T21:58:02.941ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = dfb404ab-acb8-43de-afe8-b429af0c161d dns_zone_name = oxide.test found_so_far = 0 version = 2
128742025-08-26T21:58:02.941ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
128752025-08-26T21:58:02.941ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
128762025-08-26T21:58:02.941ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
128772025-08-26T21:58:02.942ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
128782025-08-26T21:58:02.943ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
128792025-08-26T21:58:02.943ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
128802025-08-26T21:58:02.943ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
128812025-08-26T21:58:02.943ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
128822025-08-26T21:58:02.943ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 75.147914ms iteration = 27 status = {"physical_disks_added":0}
128832025-08-26T21:58:02.943ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
128842025-08-26T21:58:02.944ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
128852025-08-26T21:58:02.944ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
128862025-08-26T21:58:02.945ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1586, tv_nsec: 804957310 } })
128872025-08-26T21:58:02.945ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
128882025-08-26T21:58:02.946ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1205 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
128892025-08-26T21:58:02.946ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: dd0c655a-95c2-48fc-b4c3-9733ed4b0faa, lookup_type: ById(dd0c655a-95c2-48fc-b4c3-9733ed4b0faa) } result = Ok(())
128902025-08-26T21:58:02.946ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
128912025-08-26T21:58:02.946ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
128922025-08-26T21:58:02.946ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
128932025-08-26T21:58:02.946ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
128942025-08-26T21:58:02.947ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
128952025-08-26T21:58:02.948ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
128962025-08-26T21:58:02.948ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
128972025-08-26T21:58:02.948ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
128982025-08-26T21:58:02.949ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 80.268389ms iteration = 25 status = 0
128992025-08-26T21:58:02.949ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = dfb404ab-acb8-43de-afe8-b429af0c161d dns_zone_name = oxide.test found_so_far = 4 version = 2
129002025-08-26T21:58:02.949ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
129012025-08-26T21:58:02.949ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2025-08-26 21:46:05.296205 UTC dns_group = external generation = 2
129022025-08-26T21:58:02.949ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 116.524597ms iteration = 13 status = {"generation":2}
129032025-08-26T21:58:02.976ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
129042025-08-26T21:58:02.978ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
129052025-08-26T21:58:02.978ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
129062025-08-26T21:58:02.978ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 109.640111ms iteration = 25 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
129072025-08-26T21:58:02.983ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 114.479721ms iteration = 25 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
129082025-08-26T21:58:02.991ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1213 zone_address = fd00:1122:3344:101::2
129092025-08-26T21:58:02.991ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
129102025-08-26T21:58:02.991ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "8537d43a-2a19-4028-aadd-8045bb6e57eb", "content-length": "26", "date": "Tue, 26 Aug 2025 21:58:02 GMT"} })
129112025-08-26T21:58:02.991ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1216 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
129122025-08-26T21:58:02.993ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
129132025-08-26T21:58:02.999ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:407
129142025-08-26T21:58:02.999ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 130.357788ms iteration = 25 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
129152025-08-26T21:58:03.010ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
129162025-08-26T21:58:03.011ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
129172025-08-26T21:58:03.015ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
129182025-08-26T21:58:03.015ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
129192025-08-26T21:58:03.023ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
129202025-08-26T21:58:03.023ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
129212025-08-26T21:58:03.025ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
129222025-08-26T21:58:03.027ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
129232025-08-26T21:58:03.027ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
129242025-08-26T21:58:03.027ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
129252025-08-26T21:58:03.029ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
129262025-08-26T21:58:03.031ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
129272025-08-26T21:58:03.031ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
129282025-08-26T21:58:03.031ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
129292025-08-26T21:58:03.114ZINFOnexus (BackgroundTasks): instance reincarnation completed successfully background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:113 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0
129302025-08-26T21:58:03.114ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 255.124041ms iteration = 13 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
129312025-08-26T21:58:03.126ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
129322025-08-26T21:58:03.141ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1213 zone_address = fd00:1122:3344:101::2
129332025-08-26T21:58:03.141ZDEBGnexus (MgsClient): client request background_task = nat_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
129342025-08-26T21:58:03.141ZDEBGnexus (MgsClient): client response background_task = nat_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "802bc8ca-7af0-49f0-b9a0-c13e2735e617", "content-length": "26", "date": "Tue, 26 Aug 2025 21:58:03 GMT"} })
129352025-08-26T21:58:03.141ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1216 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
129362025-08-26T21:58:03.143ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
129372025-08-26T21:58:03.228ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 396.424696ms iteration = 25 status = {}
129382025-08-26T21:58:03.247ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
129392025-08-26T21:58:03.248ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "bc58ccb7-affd-4853-8cff-096b39584a19", "content-length": "691", "date": "Tue, 26 Aug 2025 21:58:03 GMT"} })
129402025-08-26T21:58:03.248ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
129412025-08-26T21:58:03.248ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
129422025-08-26T21:58:03.248ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
129432025-08-26T21:58:03.249ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 381.09047ms iteration = 25 status = {}
129442025-08-26T21:58:03.257ZINFOnexus (BackgroundTasks): VPC route manager sled dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
129452025-08-26T21:58:03.257ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
129462025-08-26T21:58:03.258ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "d69f8f6c-fd86-4bc2-94ea-042fafb85bda", "content-length": "372", "date": "Tue, 26 Aug 2025 21:58:03 GMT"} })
129472025-08-26T21:58:03.258ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
129482025-08-26T21:58:03.258ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
129492025-08-26T21:58:03.270ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1213 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 zone_address = fd00:1122:3344:101::2
129502025-08-26T21:58:03.270ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
129512025-08-26T21:58:03.270ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "c254fe8b-079e-463e-b55b-908224d2ccde", "content-length": "26", "date": "Tue, 26 Aug 2025 21:58:02 GMT"} })
129522025-08-26T21:58:03.270ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1216 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
129532025-08-26T21:58:03.272ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
129542025-08-26T21:58:03.277ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
129552025-08-26T21:58:03.277ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
129562025-08-26T21:58:03.277ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "1e4748e8-d02a-431f-b360-59d0b05fc069", "date": "Tue, 26 Aug 2025 21:58:03 GMT"} })
129572025-08-26T21:58:03.277ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
129582025-08-26T21:58:03.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
129592025-08-26T21:58:03.279ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
129602025-08-26T21:58:03.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
129612025-08-26T21:58:03.282ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
129622025-08-26T21:58:03.283ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
129632025-08-26T21:58:03.283ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
129642025-08-26T21:58:03.286ZINFOnexus (BackgroundTasks): VPC route manager sled dd0c655a-95c2-48fc-b4c3-9733ed4b0faa push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
129652025-08-26T21:58:03.286ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
129662025-08-26T21:58:03.286ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "0c4a5c52-7fc2-467f-bf0a-208453c0676a", "date": "Tue, 26 Aug 2025 21:58:03 GMT"} })
129672025-08-26T21:58:03.288ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 419.907786ms iteration = 25 status = {}
129682025-08-26T21:58:03.305ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
129692025-08-26T21:58:03.305ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "9e30b0ca-50d7-48a9-aa6d-cf9ab8906ea3", "date": "Tue, 26 Aug 2025 21:58:03 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
129702025-08-26T21:58:03.305ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
129712025-08-26T21:58:03.307ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "b4186a35-292d-42f4-b028-e09fe886d3f8", "date": "Tue, 26 Aug 2025 21:58:03 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
129722025-08-26T21:58:03.307ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
129732025-08-26T21:58:03.310ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
129742025-08-26T21:58:03.310ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 477.533515ms iteration = 15 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
129752025-08-26T21:58:03.338ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
129762025-08-26T21:58:03.338ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
129772025-08-26T21:58:03.340ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 508.266137ms iteration = 25 status = null
129782025-08-26T21:58:03.365ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
129792025-08-26T21:58:03.366ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "12d8f9b0-34c1-4fd8-a185-05cccf96bc47", "date": "Tue, 26 Aug 2025 21:58:03 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
129802025-08-26T21:58:03.366ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
129812025-08-26T21:58:03.368ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "f292d431-231c-4157-ba93-6eb1d24249b1", "date": "Tue, 26 Aug 2025 21:58:03 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
129822025-08-26T21:58:03.368ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
129832025-08-26T21:58:03.370ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "2e585bef-03ad-4213-a2e6-1355d3ea83e7", "date": "Tue, 26 Aug 2025 21:58:03 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
129842025-08-26T21:58:03.370ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
129852025-08-26T21:58:03.372ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
129862025-08-26T21:58:03.372ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 540.151686ms iteration = 15 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
129872025-08-26T21:58:03.425ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
129882025-08-26T21:58:03.425ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
129892025-08-26T21:58:03.428ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: d24c5c55-3a21-4a91-aeb4-45362e822df6, rack_id: e37a46f2-f789-4ca4-9841-f8c6dd705964, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(88d84f37-a4c8-4abc-af1f-27987b0d39d9) } rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0
129902025-08-26T21:58:03.428ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
129912025-08-26T21:58:03.428ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
129922025-08-26T21:58:03.438ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1832 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
129932025-08-26T21:58:03.438ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1859 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 88d84f37-a4c8-4abc-af1f-27987b0d39d9
129942025-08-26T21:58:03.439ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
129952025-08-26T21:58:03.439ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
129962025-08-26T21:58:03.439ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
129972025-08-26T21:58:03.439ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
129982025-08-26T21:58:03.439ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
129992025-08-26T21:58:03.439ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2006 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
130002025-08-26T21:58:03.439ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:408 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
130012025-08-26T21:58:03.439ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2042 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
130022025-08-26T21:58:03.439ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
130032025-08-26T21:58:03.439ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
130042025-08-26T21:58:03.439ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
130052025-08-26T21:58:03.442ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0
130062025-08-26T21:58:03.442ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
130072025-08-26T21:58:03.549ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "b614a770-fe77-448f-aea6-02ecafb8dd16", "date": "Tue, 26 Aug 2025 21:58:03 GMT"} })
130082025-08-26T21:58:03.549ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch = Switch0
130092025-08-26T21:58:03.549ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
130102025-08-26T21:58:03.549ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "66ee59a7-0b73-4b16-8ea7-4b2ca1038f77", "content-length": "682", "date": "Tue, 26 Aug 2025 21:58:03 GMT"} })
130112025-08-26T21:58:03.549ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
130122025-08-26T21:58:03.550ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
130132025-08-26T21:58:03.553ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
130142025-08-26T21:58:03.554ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
130152025-08-26T21:58:03.557ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
130162025-08-26T21:58:03.557ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
130172025-08-26T21:58:03.559ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
130182025-08-26T21:58:03.559ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
130192025-08-26T21:58:03.567ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 699.14942ms iteration = 25 status = {}
130202025-08-26T21:58:05.352ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 270 local_addr = [fd00:1122:3344:101::c]:42016 method = GET remote_addr = [fd00:1122:3344:101::d]:37550 req_id = 791e322a-2908-4bd6-b522-f8b6d1a8ba64 response_code = 200 uri = /5d0959e5-47a6-4bf3-a52b-57950b9d900d
130212025-08-26T21:58:07.868ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 146 reason = Timeout
130222025-08-26T21:58:07.868ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
130232025-08-26T21:58:07.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
130242025-08-26T21:58:07.870ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
130252025-08-26T21:58:07.873ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
130262025-08-26T21:58:07.873ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
130272025-08-26T21:58:07.873ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
130282025-08-26T21:58:07.876ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
130292025-08-26T21:58:07.876ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 8.046142ms iteration = 146 status = {"chicken_switches_updated":false}
130302025-08-26T21:58:12.868ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 74 reason = Timeout
130312025-08-26T21:58:12.868ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
130322025-08-26T21:58:12.868ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 147 reason = Timeout
130332025-08-26T21:58:12.868ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
130342025-08-26T21:58:12.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
130352025-08-26T21:58:12.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
130362025-08-26T21:58:12.870ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
130372025-08-26T21:58:12.871ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 74 reason = Timeout
130382025-08-26T21:58:12.871ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
130392025-08-26T21:58:12.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
130402025-08-26T21:58:12.873ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
130412025-08-26T21:58:12.874ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
130422025-08-26T21:58:12.874ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
130432025-08-26T21:58:12.874ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
130442025-08-26T21:58:12.876ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
130452025-08-26T21:58:12.876ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
130462025-08-26T21:58:12.876ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
130472025-08-26T21:58:12.876ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.906369ms iteration = 147 status = {"chicken_switches_updated":false}
130482025-08-26T21:58:12.876ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
130492025-08-26T21:58:12.877ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
130502025-08-26T21:58:12.877ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
130512025-08-26T21:58:12.878ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
130522025-08-26T21:58:12.881ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
130532025-08-26T21:58:12.881ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
130542025-08-26T21:58:12.881ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
130552025-08-26T21:58:12.893ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
130562025-08-26T21:58:12.893ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 22.707522ms iteration = 74 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
130572025-08-26T21:58:12.910ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
130582025-08-26T21:58:12.910ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
130592025-08-26T21:58:12.912ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 43.551732ms iteration = 74 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
130602025-08-26T21:58:15.352ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 303 local_addr = [fd00:1122:3344:101::c]:42016 method = GET remote_addr = [fd00:1122:3344:101::d]:37550 req_id = dba542a2-00cf-43d0-aeab-d879530ae29b response_code = 200 uri = /5d0959e5-47a6-4bf3-a52b-57950b9d900d
130612025-08-26T21:58:17.868ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 148 reason = Timeout
130622025-08-26T21:58:17.868ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
130632025-08-26T21:58:17.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
130642025-08-26T21:58:17.871ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
130652025-08-26T21:58:17.874ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
130662025-08-26T21:58:17.874ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
130672025-08-26T21:58:17.874ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
130682025-08-26T21:58:17.876ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
130692025-08-26T21:58:17.876ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 8.276557ms iteration = 148 status = {"chicken_switches_updated":false}
130702025-08-26T21:58:22.868ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 75 reason = Timeout
130712025-08-26T21:58:22.868ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
130722025-08-26T21:58:22.868ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 149 reason = Timeout
130732025-08-26T21:58:22.868ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
130742025-08-26T21:58:22.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
130752025-08-26T21:58:22.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
130762025-08-26T21:58:22.870ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
130772025-08-26T21:58:22.870ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
130782025-08-26T21:58:22.870ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 75 reason = Timeout
130792025-08-26T21:58:22.870ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
130802025-08-26T21:58:22.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
130812025-08-26T21:58:22.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
130822025-08-26T21:58:22.872ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
130832025-08-26T21:58:22.872ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.268156ms iteration = 75 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
130842025-08-26T21:58:22.874ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
130852025-08-26T21:58:22.874ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
130862025-08-26T21:58:22.874ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
130872025-08-26T21:58:22.876ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
130882025-08-26T21:58:22.876ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
130892025-08-26T21:58:22.876ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
130902025-08-26T21:58:22.876ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
130912025-08-26T21:58:22.876ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.804396ms iteration = 149 status = {"chicken_switches_updated":false}
130922025-08-26T21:58:22.878ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
130932025-08-26T21:58:22.878ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
130942025-08-26T21:58:22.880ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
130952025-08-26T21:58:22.883ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
130962025-08-26T21:58:22.883ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
130972025-08-26T21:58:22.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
130982025-08-26T21:58:22.899ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
130992025-08-26T21:58:22.899ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 28.339829ms iteration = 75 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
131002025-08-26T21:58:25.352ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 482 local_addr = [fd00:1122:3344:101::c]:42016 method = GET remote_addr = [fd00:1122:3344:101::d]:37550 req_id = 1cdc50ec-fbc3-4cd6-8d8b-9412420ad0f7 response_code = 200 uri = /5d0959e5-47a6-4bf3-a52b-57950b9d900d
131012025-08-26T21:58:27.868ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 150 reason = Timeout
131022025-08-26T21:58:27.868ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
131032025-08-26T21:58:27.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
131042025-08-26T21:58:27.870ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
131052025-08-26T21:58:27.874ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
131062025-08-26T21:58:27.874ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
131072025-08-26T21:58:27.874ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
131082025-08-26T21:58:27.876ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
131092025-08-26T21:58:27.876ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.675133ms iteration = 150 status = {"chicken_switches_updated":false}
131102025-08-26T21:58:32.832ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 26 reason = Timeout
131112025-08-26T21:58:32.832ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 26 reason = Timeout
131122025-08-26T21:58:32.832ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
131132025-08-26T21:58:32.832ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 26 reason = Timeout
131142025-08-26T21:58:32.832ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
131152025-08-26T21:58:32.833ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
131162025-08-26T21:58:32.833ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
131172025-08-26T21:58:32.835ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.037399ms iteration = 26 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
131182025-08-26T21:58:32.836ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1616, tv_nsec: 695416751 } })
131192025-08-26T21:58:32.836ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1616, tv_nsec: 695598635 } })
131202025-08-26T21:58:32.836ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1205
131212025-08-26T21:58:32.866ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1213 zone_address = fd00:1122:3344:101::2
131222025-08-26T21:58:32.866ZDEBGnexus (MgsClient): client request background_task = nat_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
131232025-08-26T21:58:32.866ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1205
131242025-08-26T21:58:32.867ZDEBGnexus (MgsClient): client response background_task = nat_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "528bca9e-d6a4-4e3d-a8ea-e03e6107af68", "content-length": "26", "date": "Tue, 26 Aug 2025 21:58:32 GMT"} })
131252025-08-26T21:58:32.867ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1216 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
131262025-08-26T21:58:32.867ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
131272025-08-26T21:58:32.867ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 26 reason = Timeout
131282025-08-26T21:58:32.868ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 26 reason = Timeout
131292025-08-26T21:58:32.868ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 26 reason = Timeout
131302025-08-26T21:58:32.868ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
131312025-08-26T21:58:32.868ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
131322025-08-26T21:58:32.868ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
131332025-08-26T21:58:32.868ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 26 reason = Timeout
131342025-08-26T21:58:32.868ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 12.28\u{b5}s iteration = 26 status = {"disabled":true,"errors":[],"sps":[]}
131352025-08-26T21:58:32.868ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 26 reason = Timeout
131362025-08-26T21:58:32.868ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 26 reason = Timeout
131372025-08-26T21:58:32.868ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
131382025-08-26T21:58:32.868ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
131392025-08-26T21:58:32.868ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 28 reason = Timeout
131402025-08-26T21:58:32.868ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
131412025-08-26T21:58:32.868ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 26 reason = Timeout
131422025-08-26T21:58:32.868ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
131432025-08-26T21:58:32.868ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
131442025-08-26T21:58:32.868ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 76 reason = Timeout
131452025-08-26T21:58:32.868ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
131462025-08-26T21:58:32.868ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 151 reason = Timeout
131472025-08-26T21:58:32.868ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
131482025-08-26T21:58:32.868ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 26 reason = Timeout
131492025-08-26T21:58:32.868ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
131502025-08-26T21:58:32.868ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 26 reason = Timeout
131512025-08-26T21:58:32.868ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
131522025-08-26T21:58:32.868ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 26 reason = Timeout
131532025-08-26T21:58:32.868ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
131542025-08-26T21:58:32.868ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 26 reason = Timeout
131552025-08-26T21:58:32.868ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
131562025-08-26T21:58:32.868ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 26 reason = Timeout
131572025-08-26T21:58:32.868ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 26 reason = Timeout
131582025-08-26T21:58:32.868ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
131592025-08-26T21:58:32.868ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
131602025-08-26T21:58:32.868ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 26 reason = Timeout
131612025-08-26T21:58:32.868ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
131622025-08-26T21:58:32.868ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
131632025-08-26T21:58:32.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
131642025-08-26T21:58:32.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
131652025-08-26T21:58:32.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
131662025-08-26T21:58:32.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
131672025-08-26T21:58:32.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
131682025-08-26T21:58:32.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
131692025-08-26T21:58:32.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
131702025-08-26T21:58:32.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
131712025-08-26T21:58:32.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
131722025-08-26T21:58:32.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
131732025-08-26T21:58:32.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
131742025-08-26T21:58:32.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
131752025-08-26T21:58:32.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
131762025-08-26T21:58:32.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
131772025-08-26T21:58:32.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
131782025-08-26T21:58:32.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
131792025-08-26T21:58:32.870ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 2.092058ms iteration = 26 status = {"errors":[],"finish_invoked_ok":[]}
131802025-08-26T21:58:32.870ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
131812025-08-26T21:58:32.870ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
131822025-08-26T21:58:32.870ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
131832025-08-26T21:58:32.870ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
131842025-08-26T21:58:32.870ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
131852025-08-26T21:58:32.870ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
131862025-08-26T21:58:32.870ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
131872025-08-26T21:58:32.871ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 76 reason = Timeout
131882025-08-26T21:58:32.871ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
131892025-08-26T21:58:32.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
131902025-08-26T21:58:32.873ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
131912025-08-26T21:58:32.875ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
131922025-08-26T21:58:32.875ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
131932025-08-26T21:58:32.875ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
131942025-08-26T21:58:32.875ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
131952025-08-26T21:58:32.875ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
131962025-08-26T21:58:32.876ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
131972025-08-26T21:58:32.876ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
131982025-08-26T21:58:32.876ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
131992025-08-26T21:58:32.876ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
132002025-08-26T21:58:32.876ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
132012025-08-26T21:58:32.876ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
132022025-08-26T21:58:32.876ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
132032025-08-26T21:58:32.876ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
132042025-08-26T21:58:32.876ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
132052025-08-26T21:58:32.876ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
132062025-08-26T21:58:32.877ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
132072025-08-26T21:58:32.877ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
132082025-08-26T21:58:32.877ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
132092025-08-26T21:58:32.877ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
132102025-08-26T21:58:32.877ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
132112025-08-26T21:58:32.878ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
132122025-08-26T21:58:32.878ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
132132025-08-26T21:58:32.878ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
132142025-08-26T21:58:32.879ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
132152025-08-26T21:58:32.879ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
132162025-08-26T21:58:32.879ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
132172025-08-26T21:58:32.879ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
132182025-08-26T21:58:32.880ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
132192025-08-26T21:58:32.880ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
132202025-08-26T21:58:32.880ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
132212025-08-26T21:58:32.881ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
132222025-08-26T21:58:32.881ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
132232025-08-26T21:58:32.881ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 13.251609ms iteration = 151 status = {"chicken_switches_updated":false}
132242025-08-26T21:58:32.881ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1616, tv_nsec: 740931347 } })
132252025-08-26T21:58:32.881ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
132262025-08-26T21:58:32.881ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1205 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
132272025-08-26T21:58:32.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
132282025-08-26T21:58:32.882ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
132292025-08-26T21:58:32.882ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
132302025-08-26T21:58:32.883ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
132312025-08-26T21:58:32.883ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
132322025-08-26T21:58:32.883ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
132332025-08-26T21:58:32.884ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
132342025-08-26T21:58:32.885ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
132352025-08-26T21:58:32.885ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
132362025-08-26T21:58:32.885ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
132372025-08-26T21:58:32.887ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
132382025-08-26T21:58:32.887ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
132392025-08-26T21:58:32.887ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
132402025-08-26T21:58:32.887ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:407
132412025-08-26T21:58:32.887ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 19.572991ms iteration = 26 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
132422025-08-26T21:58:32.887ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
132432025-08-26T21:58:32.887ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
132442025-08-26T21:58:32.888ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
132452025-08-26T21:58:32.898ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 30.719352ms iteration = 28 status = {"physical_disks_added":0}
132462025-08-26T21:58:32.910ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
132472025-08-26T21:58:32.910ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 38.899827ms iteration = 76 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
132482025-08-26T21:58:32.916ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
132492025-08-26T21:58:32.916ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
132502025-08-26T21:58:32.920ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 52.106105ms iteration = 26 status = {"errors":[],"requests_created_ok":[]}
132512025-08-26T21:58:32.920ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
132522025-08-26T21:58:32.920ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
132532025-08-26T21:58:32.921ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
132542025-08-26T21:58:32.921ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
132552025-08-26T21:58:32.921ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
132562025-08-26T21:58:32.921ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
132572025-08-26T21:58:32.921ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 53.67996ms iteration = 76 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
132582025-08-26T21:58:32.923ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
132592025-08-26T21:58:32.923ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
132602025-08-26T21:58:32.924ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 56.78242ms iteration = 26 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
132612025-08-26T21:58:32.929ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
132622025-08-26T21:58:32.930ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
132632025-08-26T21:58:32.931ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
132642025-08-26T21:58:32.931ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
132652025-08-26T21:58:32.932ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:550 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
132662025-08-26T21:58:32.932ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 64.061254ms iteration = 26 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
132672025-08-26T21:58:32.932ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 64.029533ms iteration = 26 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
132682025-08-26T21:58:32.933ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 65.165469ms iteration = 26 status = {"errors":[],"garbage_collect_requested":[]}
132692025-08-26T21:58:32.933ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
132702025-08-26T21:58:32.933ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
132712025-08-26T21:58:32.935ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
132722025-08-26T21:58:32.937ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: dd0c655a-95c2-48fc-b4c3-9733ed4b0faa, lookup_type: ById(dd0c655a-95c2-48fc-b4c3-9733ed4b0faa) } result = Ok(())
132732025-08-26T21:58:32.937ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
132742025-08-26T21:58:32.937ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
132752025-08-26T21:58:32.939ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 71.336118ms iteration = 26 status = 0
132762025-08-26T21:58:32.950ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
132772025-08-26T21:58:32.950ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
132782025-08-26T21:58:32.951ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 83.502282ms iteration = 26 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
132792025-08-26T21:58:32.975ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
132802025-08-26T21:58:32.975ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
132812025-08-26T21:58:32.975ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
132822025-08-26T21:58:32.975ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
132832025-08-26T21:58:32.975ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
132842025-08-26T21:58:32.975ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
132852025-08-26T21:58:32.977ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 144.278552ms iteration = 26 status = null
132862025-08-26T21:58:32.983ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1213 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 zone_address = fd00:1122:3344:101::2
132872025-08-26T21:58:32.983ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
132882025-08-26T21:58:32.983ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "3f2238a6-954f-4d4b-a435-4e03659685e0", "content-length": "26", "date": "Tue, 26 Aug 2025 21:58:32 GMT"} })
132892025-08-26T21:58:32.983ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1216 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
132902025-08-26T21:58:32.985ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
132912025-08-26T21:58:33.004ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1213 zone_address = fd00:1122:3344:101::2
132922025-08-26T21:58:33.004ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
132932025-08-26T21:58:33.004ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "99f3ce7b-bde6-4aba-a317-ff15d8798dc3", "content-length": "26", "date": "Tue, 26 Aug 2025 21:58:32 GMT"} })
132942025-08-26T21:58:33.004ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1216 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
132952025-08-26T21:58:33.006ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
132962025-08-26T21:58:33.014ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
132972025-08-26T21:58:33.014ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
132982025-08-26T21:58:33.014ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
132992025-08-26T21:58:33.014ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
133002025-08-26T21:58:33.014ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 146.222136ms iteration = 26 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
133012025-08-26T21:58:33.048ZINFOnexus (BackgroundTasks): VPC route manager sled dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
133022025-08-26T21:58:33.048ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
133032025-08-26T21:58:33.048ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "0e760edf-6a23-4c7d-822b-3b593ec70181", "content-length": "372", "date": "Tue, 26 Aug 2025 21:58:32 GMT"} })
133042025-08-26T21:58:33.049ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
133052025-08-26T21:58:33.049ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
133062025-08-26T21:58:33.056ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
133072025-08-26T21:58:33.057ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
133082025-08-26T21:58:33.058ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
133092025-08-26T21:58:33.061ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
133102025-08-26T21:58:33.061ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
133112025-08-26T21:58:33.061ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
133122025-08-26T21:58:33.062ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
133132025-08-26T21:58:33.065ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
133142025-08-26T21:58:33.065ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
133152025-08-26T21:58:33.065ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
133162025-08-26T21:58:33.083ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 251.111281ms iteration = 26 status = {}
133172025-08-26T21:58:33.098ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
133182025-08-26T21:58:33.098ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
133192025-08-26T21:58:33.098ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "72fdb205-790b-40cf-8c62-edd28ce8b328", "date": "Tue, 26 Aug 2025 21:58:33 GMT"} })
133202025-08-26T21:58:33.098ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
133212025-08-26T21:58:33.099ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
133222025-08-26T21:58:33.100ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
133232025-08-26T21:58:33.100ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
133242025-08-26T21:58:33.104ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
133252025-08-26T21:58:33.104ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
133262025-08-26T21:58:33.107ZINFOnexus (BackgroundTasks): VPC route manager sled dd0c655a-95c2-48fc-b4c3-9733ed4b0faa push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
133272025-08-26T21:58:33.107ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
133282025-08-26T21:58:33.108ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "bbdac8eb-3da6-4b11-9ee3-1addca22864a", "date": "Tue, 26 Aug 2025 21:58:33 GMT"} })
133292025-08-26T21:58:33.110ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 242.085247ms iteration = 26 status = {}
133302025-08-26T21:58:33.135ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
133312025-08-26T21:58:33.136ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "fc0f0c4c-4425-4dd4-9d1f-405e7702c3dd", "content-length": "691", "date": "Tue, 26 Aug 2025 21:58:33 GMT"} })
133322025-08-26T21:58:33.136ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
133332025-08-26T21:58:33.136ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
133342025-08-26T21:58:33.136ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
133352025-08-26T21:58:33.138ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 270.15975ms iteration = 26 status = {}
133362025-08-26T21:58:33.167ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
133372025-08-26T21:58:33.167ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
133382025-08-26T21:58:33.169ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: d24c5c55-3a21-4a91-aeb4-45362e822df6, rack_id: e37a46f2-f789-4ca4-9841-f8c6dd705964, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(88d84f37-a4c8-4abc-af1f-27987b0d39d9) } rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0
133392025-08-26T21:58:33.169ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
133402025-08-26T21:58:33.169ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
133412025-08-26T21:58:33.181ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1832 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
133422025-08-26T21:58:33.181ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1859 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 88d84f37-a4c8-4abc-af1f-27987b0d39d9
133432025-08-26T21:58:33.182ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
133442025-08-26T21:58:33.182ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
133452025-08-26T21:58:33.182ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
133462025-08-26T21:58:33.182ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
133472025-08-26T21:58:33.182ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
133482025-08-26T21:58:33.182ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2006 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
133492025-08-26T21:58:33.182ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:408 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
133502025-08-26T21:58:33.182ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2042 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
133512025-08-26T21:58:33.182ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
133522025-08-26T21:58:33.182ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
133532025-08-26T21:58:33.183ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
133542025-08-26T21:58:33.185ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0
133552025-08-26T21:58:33.185ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
133562025-08-26T21:58:33.289ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "c65735ba-5514-491c-a112-0cb33b0f1985", "date": "Tue, 26 Aug 2025 21:58:33 GMT"} })
133572025-08-26T21:58:33.289ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch = Switch0
133582025-08-26T21:58:33.289ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
133592025-08-26T21:58:33.289ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "53e192b4-8b62-47ca-a6d7-fbd0aa9ab860", "content-length": "682", "date": "Tue, 26 Aug 2025 21:58:33 GMT"} })
133602025-08-26T21:58:33.289ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
133612025-08-26T21:58:33.290ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
133622025-08-26T21:58:33.294ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
133632025-08-26T21:58:33.294ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
133642025-08-26T21:58:33.297ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
133652025-08-26T21:58:33.298ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
133662025-08-26T21:58:33.299ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
133672025-08-26T21:58:33.300ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
133682025-08-26T21:58:33.307ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 439.20803ms iteration = 26 status = {}
133692025-08-26T21:58:35.351ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 317 local_addr = [fd00:1122:3344:101::c]:42016 method = GET remote_addr = [fd00:1122:3344:101::d]:37550 req_id = 35530546-89e9-46b8-a571-dc3ba2038a86 response_code = 200 uri = /5d0959e5-47a6-4bf3-a52b-57950b9d900d
133702025-08-26T21:58:37.868ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 152 reason = Timeout
133712025-08-26T21:58:37.868ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
133722025-08-26T21:58:37.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
133732025-08-26T21:58:37.870ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
133742025-08-26T21:58:37.873ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
133752025-08-26T21:58:37.873ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
133762025-08-26T21:58:37.874ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
133772025-08-26T21:58:37.876ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
133782025-08-26T21:58:37.876ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.831726ms iteration = 152 status = {"chicken_switches_updated":false}
133792025-08-26T21:58:42.868ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 77 reason = Timeout
133802025-08-26T21:58:42.868ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
133812025-08-26T21:58:42.868ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 153 reason = Timeout
133822025-08-26T21:58:42.868ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
133832025-08-26T21:58:42.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
133842025-08-26T21:58:42.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
133852025-08-26T21:58:42.870ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
133862025-08-26T21:58:42.870ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 77 reason = Timeout
133872025-08-26T21:58:42.870ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
133882025-08-26T21:58:42.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
133892025-08-26T21:58:42.873ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
133902025-08-26T21:58:42.874ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
133912025-08-26T21:58:42.874ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
133922025-08-26T21:58:42.874ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
133932025-08-26T21:58:42.876ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
133942025-08-26T21:58:42.876ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.639352ms iteration = 153 status = {"chicken_switches_updated":false}
133952025-08-26T21:58:42.876ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
133962025-08-26T21:58:42.876ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
133972025-08-26T21:58:42.876ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
133982025-08-26T21:58:42.878ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
133992025-08-26T21:58:42.878ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
134002025-08-26T21:58:42.880ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
134012025-08-26T21:58:42.883ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
134022025-08-26T21:58:42.883ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
134032025-08-26T21:58:42.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
134042025-08-26T21:58:42.900ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
134052025-08-26T21:58:42.900ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 29.858362ms iteration = 77 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
134062025-08-26T21:58:42.902ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
134072025-08-26T21:58:42.902ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
134082025-08-26T21:58:42.939ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 70.773595ms iteration = 77 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
134092025-08-26T21:58:45.351ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 279 local_addr = [fd00:1122:3344:101::c]:42016 method = GET remote_addr = [fd00:1122:3344:101::d]:37550 req_id = 0000ebd5-8563-48d7-b21a-bf9dc7fe956e response_code = 200 uri = /5d0959e5-47a6-4bf3-a52b-57950b9d900d
134102025-08-26T21:58:47.868ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 154 reason = Timeout
134112025-08-26T21:58:47.868ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
134122025-08-26T21:58:47.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
134132025-08-26T21:58:47.871ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
134142025-08-26T21:58:47.874ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
134152025-08-26T21:58:47.874ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
134162025-08-26T21:58:47.874ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
134172025-08-26T21:58:47.876ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
134182025-08-26T21:58:47.876ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 8.518502ms iteration = 154 status = {"chicken_switches_updated":false}
134192025-08-26T21:58:52.868ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 78 reason = Timeout
134202025-08-26T21:58:52.868ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
134212025-08-26T21:58:52.868ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 155 reason = Timeout
134222025-08-26T21:58:52.868ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
134232025-08-26T21:58:52.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
134242025-08-26T21:58:52.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
134252025-08-26T21:58:52.870ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
134262025-08-26T21:58:52.870ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
134272025-08-26T21:58:52.870ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 78 reason = Timeout
134282025-08-26T21:58:52.870ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
134292025-08-26T21:58:52.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
134302025-08-26T21:58:52.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
134312025-08-26T21:58:52.872ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.779525ms iteration = 78 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
134322025-08-26T21:58:52.873ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
134332025-08-26T21:58:52.873ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
134342025-08-26T21:58:52.873ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
134352025-08-26T21:58:52.874ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
134362025-08-26T21:58:52.876ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
134372025-08-26T21:58:52.876ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.165882ms iteration = 155 status = {"chicken_switches_updated":false}
134382025-08-26T21:58:52.876ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
134392025-08-26T21:58:52.876ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
134402025-08-26T21:58:52.876ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
134412025-08-26T21:58:52.877ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
134422025-08-26T21:58:52.877ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
134432025-08-26T21:58:52.878ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
134442025-08-26T21:58:52.881ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
134452025-08-26T21:58:52.881ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
134462025-08-26T21:58:52.881ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
134472025-08-26T21:58:52.893ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
134482025-08-26T21:58:52.893ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 22.349204ms iteration = 78 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
134492025-08-26T21:58:55.351ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 477 local_addr = [fd00:1122:3344:101::c]:42016 method = GET remote_addr = [fd00:1122:3344:101::d]:37550 req_id = be467fd8-8835-4af8-a14e-ce78da4eeeb3 response_code = 200 uri = /5d0959e5-47a6-4bf3-a52b-57950b9d900d
134502025-08-26T21:58:57.868ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 156 reason = Timeout
134512025-08-26T21:58:57.868ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
134522025-08-26T21:58:57.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
134532025-08-26T21:58:57.870ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
134542025-08-26T21:58:57.873ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
134552025-08-26T21:58:57.873ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
134562025-08-26T21:58:57.873ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
134572025-08-26T21:58:57.875ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
134582025-08-26T21:58:57.875ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.152321ms iteration = 156 status = {"chicken_switches_updated":false}
134592025-08-26T21:59:00.345ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:1025 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::d]:55213
134602025-08-26T21:59:00.345ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:55213 req_id = 3016c9e8-c8ec-48b8-b3aa-41298ac93b33 roles = RoleSet { roles: {} } uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?sort_by=id_ascending
134612025-08-26T21:59:00.345ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:55213 req_id = 3016c9e8-c8ec-48b8-b3aa-41298ac93b33 resource = Database result = Ok(()) uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?sort_by=id_ascending
134622025-08-26T21:59:00.348ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 3602 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:55213 req_id = 3016c9e8-c8ec-48b8-b3aa-41298ac93b33 response_code = 200 uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?sort_by=id_ascending
134632025-08-26T21:59:00.348ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:55213 req_id = 45338a75-3e2f-4aa3-a660-8e1d58da84e6 roles = RoleSet { roles: {} } uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMzhjYTg4YS1lZGE1LTQwZGEtOGE4MS1hN2UzMzdhMmM4ZmIifX0%3D
134642025-08-26T21:59:00.349ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:55213 req_id = 45338a75-3e2f-4aa3-a660-8e1d58da84e6 resource = Database result = Ok(()) uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMzhjYTg4YS1lZGE1LTQwZGEtOGE4MS1hN2UzMzdhMmM4ZmIifX0%3D
134652025-08-26T21:59:00.351ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 2442 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:55213 req_id = 45338a75-3e2f-4aa3-a660-8e1d58da84e6 response_code = 200 uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMzhjYTg4YS1lZGE1LTQwZGEtOGE4MS1hN2UzMzdhMmM4ZmIifX0%3D
134662025-08-26T21:59:02.832ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 14 reason = Timeout
134672025-08-26T21:59:02.832ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 14 reason = Timeout
134682025-08-26T21:59:02.832ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
134692025-08-26T21:59:02.832ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
134702025-08-26T21:59:02.832ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 27 reason = Timeout
134712025-08-26T21:59:02.832ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 27 reason = Timeout
134722025-08-26T21:59:02.832ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
134732025-08-26T21:59:02.832ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 27 reason = Timeout
134742025-08-26T21:59:02.832ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 16 reason = Timeout
134752025-08-26T21:59:02.832ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
134762025-08-26T21:59:02.832ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 16 reason = Timeout
134772025-08-26T21:59:02.832ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 14 reason = Timeout
134782025-08-26T21:59:02.832ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-08-26 21:49:02.832694800 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
134792025-08-26T21:59:02.832ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
134802025-08-26T21:59:02.832ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 14 reason = Timeout
134812025-08-26T21:59:02.832ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 14 reason = Timeout
134822025-08-26T21:59:02.832ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 14 reason = Timeout
134832025-08-26T21:59:02.832ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
134842025-08-26T21:59:02.832ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
134852025-08-26T21:59:02.832ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
134862025-08-26T21:59:02.832ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
134872025-08-26T21:59:02.833ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
134882025-08-26T21:59:02.833ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
134892025-08-26T21:59:02.833ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
134902025-08-26T21:59:02.833ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
134912025-08-26T21:59:02.833ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("60abf731-31e1-4111-a3e2-65da89eaed87.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("84c849eb-0949-41d9-ad02-9a5509144564.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("84c849eb-0949-41d9-ad02-9a5509144564.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::8)) }], valid_until: Instant { tv_sec: 1646, tv_nsec: 692545556 } })
134922025-08-26T21:59:02.833ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
134932025-08-26T21:59:02.833ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("852fd1a3-de9a-47e8-a667-5e8dab880ad3.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("c68f1501-294c-4874-92ca-8fc0546a5870.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("d741a72f-8745-4723-9e80-e1e72987266b.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("d741a72f-8745-4723-9e80-e1e72987266b.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:3::1)) }], valid_until: Instant { tv_sec: 1646, tv_nsec: 692730041 } })
134942025-08-26T21:59:02.833ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
134952025-08-26T21:59:02.833ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.164786ms iteration = 14 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
134962025-08-26T21:59:02.833ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
134972025-08-26T21:59:02.833ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } dns_group = internal
134982025-08-26T21:59:02.833ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.166606ms iteration = 14 status = {"addresses":["[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:3::1]:5353"]}
134992025-08-26T21:59:02.833ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
135002025-08-26T21:59:02.834ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
135012025-08-26T21:59:02.834ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
135022025-08-26T21:59:02.834ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 15 reason = Timeout
135032025-08-26T21:59:02.834ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:91 reason = blueprint disabled target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8
135042025-08-26T21:59:02.834ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 14.07\u{b5}s iteration = 15 status = {"enabled":false,"target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8"}
135052025-08-26T21:59:02.834ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
135062025-08-26T21:59:02.835ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1646, tv_nsec: 694558412 } })
135072025-08-26T21:59:02.835ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1205
135082025-08-26T21:59:02.836ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-08-26 21:49:02.832694800 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
135092025-08-26T21:59:02.836ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 3.315645ms iteration = 14 status = {"expiration":"2025-08-26T21:49:02.832694800Z","pruned":[]}
135102025-08-26T21:59:02.836ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.99262ms iteration = 27 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
135112025-08-26T21:59:02.836ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
135122025-08-26T21:59:02.836ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
135132025-08-26T21:59:02.837ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
135142025-08-26T21:59:02.838ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1646, tv_nsec: 697544699 } })
135152025-08-26T21:59:02.838ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1205
135162025-08-26T21:59:02.839ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
135172025-08-26T21:59:02.839ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
135182025-08-26T21:59:02.841ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
135192025-08-26T21:59:02.841ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
135202025-08-26T21:59:02.845ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
135212025-08-26T21:59:02.845ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
135222025-08-26T21:59:02.845ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
135232025-08-26T21:59:02.846ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
135242025-08-26T21:59:02.846ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
135252025-08-26T21:59:02.846ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
135262025-08-26T21:59:02.846ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
135272025-08-26T21:59:02.846ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
135282025-08-26T21:59:02.847ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
135292025-08-26T21:59:02.848ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
135302025-08-26T21:59:02.853ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
135312025-08-26T21:59:02.853ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
135322025-08-26T21:59:02.853ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
135332025-08-26T21:59:02.859ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 17 reason = Timeout
135342025-08-26T21:59:02.859ZDEBGnexus (BackgroundTasks): blueprint planning disabled, doing nothing background_task = blueprint_planner
135352025-08-26T21:59:02.859ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 14 reason = Timeout
135362025-08-26T21:59:02.859ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 11.84\u{b5}s iteration = 17 status = "Disabled"
135372025-08-26T21:59:02.859ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:53
135382025-08-26T21:59:02.859ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 14 reason = Timeout
135392025-08-26T21:59:02.859ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 14 reason = Timeout
135402025-08-26T21:59:02.859ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
135412025-08-26T21:59:02.859ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
135422025-08-26T21:59:02.859ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
135432025-08-26T21:59:02.859ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
135442025-08-26T21:59:02.859ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
135452025-08-26T21:59:02.859ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
135462025-08-26T21:59:02.860ZDEBGnexus (BackgroundTasks): activating background_task = webhook_deliverator iteration = 14 reason = Timeout
135472025-08-26T21:59:02.860ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 14 reason = Timeout
135482025-08-26T21:59:02.860ZDEBGnexus (BackgroundTasks): roles background_task = webhook_deliverator roles = RoleSet { roles: {} }
135492025-08-26T21:59:02.860ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
135502025-08-26T21:59:02.860ZDEBGnexus (BackgroundTasks): activating background_task = alert_dispatcher iteration = 14 reason = Timeout
135512025-08-26T21:59:02.860ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
135522025-08-26T21:59:02.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
135532025-08-26T21:59:02.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = alert_dispatcher resource = Database result = Ok(())
135542025-08-26T21:59:02.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = webhook_deliverator resource = Database result = Ok(())
135552025-08-26T21:59:02.861ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
135562025-08-26T21:59:02.863ZINFOnexus (BackgroundTasks): all webhook delivery tasks completed background_task = webhook_deliverator file = nexus/src/app/background/tasks/webhook_deliverator.rs:204 num_receivers = 0
135572025-08-26T21:59:02.863ZDEBGnexus (BackgroundTasks): activation complete background_task = webhook_deliverator elapsed = 2.502606ms iteration = 14 status = {"by_rx":{},"error":null}
135582025-08-26T21:59:02.863ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
135592025-08-26T21:59:02.863ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
135602025-08-26T21:59:02.863ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 2.590118ms iteration = 14 status = {"errors":[],"found_port_ok":[]}
135612025-08-26T21:59:02.863ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
135622025-08-26T21:59:02.863ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:170 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
135632025-08-26T21:59:02.863ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 4.095012ms iteration = 14 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
135642025-08-26T21:59:02.863ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
135652025-08-26T21:59:02.865ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
135662025-08-26T21:59:02.865ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = alert_dispatcher resource = Database result = Ok(())
135672025-08-26T21:59:02.866ZDEBGnexus (BackgroundTasks): activation complete background_task = alert_dispatcher elapsed = 5.822011ms iteration = 14 status = {"dispatched":[],"errors":[],"glob_version":"183.0.0","globs_reprocessed":{},"no_receivers":[]}
135682025-08-26T21:59:02.868ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 27 reason = Timeout
135692025-08-26T21:59:02.868ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 27 reason = Timeout
135702025-08-26T21:59:02.868ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 12.3\u{b5}s iteration = 27 status = {"disabled":true,"errors":[],"sps":[]}
135712025-08-26T21:59:02.868ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 27 reason = Timeout
135722025-08-26T21:59:02.868ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
135732025-08-26T21:59:02.868ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 27 reason = Timeout
135742025-08-26T21:59:02.868ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
135752025-08-26T21:59:02.868ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
135762025-08-26T21:59:02.868ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 27 reason = Timeout
135772025-08-26T21:59:02.868ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 27 reason = Timeout
135782025-08-26T21:59:02.868ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
135792025-08-26T21:59:02.868ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
135802025-08-26T21:59:02.868ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 27 reason = Timeout
135812025-08-26T21:59:02.868ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
135822025-08-26T21:59:02.868ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 79 reason = Timeout
135832025-08-26T21:59:02.868ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
135842025-08-26T21:59:02.868ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 157 reason = Timeout
135852025-08-26T21:59:02.868ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
135862025-08-26T21:59:02.868ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 27 reason = Timeout
135872025-08-26T21:59:02.868ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
135882025-08-26T21:59:02.868ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 27 reason = Timeout
135892025-08-26T21:59:02.868ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
135902025-08-26T21:59:02.868ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 27 reason = Timeout
135912025-08-26T21:59:02.868ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
135922025-08-26T21:59:02.868ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
135932025-08-26T21:59:02.868ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 29 reason = Timeout
135942025-08-26T21:59:02.868ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
135952025-08-26T21:59:02.868ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 27 reason = Timeout
135962025-08-26T21:59:02.868ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
135972025-08-26T21:59:02.868ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 27 reason = Timeout
135982025-08-26T21:59:02.868ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
135992025-08-26T21:59:02.868ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 27 reason = Timeout
136002025-08-26T21:59:02.868ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
136012025-08-26T21:59:02.868ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
136022025-08-26T21:59:02.868ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 27 reason = Timeout
136032025-08-26T21:59:02.868ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
136042025-08-26T21:59:02.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
136052025-08-26T21:59:02.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
136062025-08-26T21:59:02.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
136072025-08-26T21:59:02.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
136082025-08-26T21:59:02.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
136092025-08-26T21:59:02.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
136102025-08-26T21:59:02.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
136112025-08-26T21:59:02.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
136122025-08-26T21:59:02.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
136132025-08-26T21:59:02.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
136142025-08-26T21:59:02.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
136152025-08-26T21:59:02.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
136162025-08-26T21:59:02.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
136172025-08-26T21:59:02.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
136182025-08-26T21:59:02.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
136192025-08-26T21:59:02.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
136202025-08-26T21:59:02.870ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
136212025-08-26T21:59:02.870ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
136222025-08-26T21:59:02.871ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 79 reason = Timeout
136232025-08-26T21:59:02.871ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
136242025-08-26T21:59:02.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
136252025-08-26T21:59:02.872ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
136262025-08-26T21:59:02.874ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
136272025-08-26T21:59:02.874ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
136282025-08-26T21:59:02.874ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
136292025-08-26T21:59:02.874ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
136302025-08-26T21:59:02.874ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
136312025-08-26T21:59:02.874ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
136322025-08-26T21:59:02.875ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
136332025-08-26T21:59:02.875ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
136342025-08-26T21:59:02.875ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
136352025-08-26T21:59:02.875ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
136362025-08-26T21:59:02.875ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
136372025-08-26T21:59:02.876ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
136382025-08-26T21:59:02.876ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
136392025-08-26T21:59:02.876ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
136402025-08-26T21:59:02.876ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 8.501402ms iteration = 27 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
136412025-08-26T21:59:02.878ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
136422025-08-26T21:59:02.878ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
136432025-08-26T21:59:02.878ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
136442025-08-26T21:59:02.879ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
136452025-08-26T21:59:02.880ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1646, tv_nsec: 740285303 } })
136462025-08-26T21:59:02.881ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1205 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
136472025-08-26T21:59:02.881ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
136482025-08-26T21:59:02.881ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 13.125716ms iteration = 157 status = {"chicken_switches_updated":false}
136492025-08-26T21:59:02.881ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
136502025-08-26T21:59:02.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
136512025-08-26T21:59:02.882ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
136522025-08-26T21:59:02.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
136532025-08-26T21:59:02.883ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
136542025-08-26T21:59:02.891ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
136552025-08-26T21:59:02.896ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
136562025-08-26T21:59:02.896ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
136572025-08-26T21:59:02.897ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
136582025-08-26T21:59:02.906ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
136592025-08-26T21:59:02.906ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
136602025-08-26T21:59:02.910ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
136612025-08-26T21:59:02.910ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
136622025-08-26T21:59:02.910ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
136632025-08-26T21:59:02.912ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
136642025-08-26T21:59:02.912ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
136652025-08-26T21:59:02.912ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
136662025-08-26T21:59:02.913ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
136672025-08-26T21:59:02.914ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
136682025-08-26T21:59:02.915ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
136692025-08-26T21:59:02.915ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
136702025-08-26T21:59:02.915ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
136712025-08-26T21:59:02.915ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 8e720f02-dde8-4423-a6e3-91698373805a dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
136722025-08-26T21:59:02.915ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
136732025-08-26T21:59:02.915ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
136742025-08-26T21:59:02.915ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
136752025-08-26T21:59:02.917ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
136762025-08-26T21:59:02.917ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
136772025-08-26T21:59:02.917ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
136782025-08-26T21:59:02.917ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
136792025-08-26T21:59:02.921ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
136802025-08-26T21:59:02.921ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
136812025-08-26T21:59:02.921ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
136822025-08-26T21:59:02.922ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 54.495348ms iteration = 27 status = {"errors":[],"garbage_collect_requested":[]}
136832025-08-26T21:59:02.925ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
136842025-08-26T21:59:02.925ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 53.537507ms iteration = 79 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
136852025-08-26T21:59:02.925ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 8e720f02-dde8-4423-a6e3-91698373805a dns_zone_name = control-plane.oxide.internal found_so_far = 59 version = 1
136862025-08-26T21:59:02.925ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
136872025-08-26T21:59:02.925ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2025-08-26 21:46:04.065217 UTC dns_group = internal generation = 1
136882025-08-26T21:59:02.925ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 92.803842ms iteration = 14 status = {"generation":1}
136892025-08-26T21:59:02.928ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
136902025-08-26T21:59:02.928ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:138 found = 0 not_ready_to_be_deleted = 0
136912025-08-26T21:59:02.928ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 68.986745ms iteration = 14 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
136922025-08-26T21:59:02.928ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
136932025-08-26T21:59:02.929ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
136942025-08-26T21:59:02.930ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 62.377756ms iteration = 27 status = {"errors":[],"requests_created_ok":[]}
136952025-08-26T21:59:02.930ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
136962025-08-26T21:59:02.931ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 62.770835ms iteration = 27 status = {"errors":[],"finish_invoked_ok":[]}
136972025-08-26T21:59:02.931ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
136982025-08-26T21:59:02.932ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
136992025-08-26T21:59:02.932ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
137002025-08-26T21:59:02.932ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 64.634057ms iteration = 79 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
137012025-08-26T21:59:02.932ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
137022025-08-26T21:59:02.932ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
137032025-08-26T21:59:02.933ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
137042025-08-26T21:59:02.934ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
137052025-08-26T21:59:02.934ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
137062025-08-26T21:59:02.934ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
137072025-08-26T21:59:02.934ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
137082025-08-26T21:59:02.934ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
137092025-08-26T21:59:02.935ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
137102025-08-26T21:59:02.935ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
137112025-08-26T21:59:02.935ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
137122025-08-26T21:59:02.935ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
137132025-08-26T21:59:02.935ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
137142025-08-26T21:59:02.935ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
137152025-08-26T21:59:02.936ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
137162025-08-26T21:59:02.936ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 67.993042ms iteration = 27 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
137172025-08-26T21:59:02.937ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
137182025-08-26T21:59:02.937ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
137192025-08-26T21:59:02.937ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
137202025-08-26T21:59:02.937ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
137212025-08-26T21:59:02.937ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
137222025-08-26T21:59:02.937ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
137232025-08-26T21:59:02.937ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
137242025-08-26T21:59:02.938ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
137252025-08-26T21:59:02.938ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
137262025-08-26T21:59:02.938ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
137272025-08-26T21:59:02.939ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
137282025-08-26T21:59:02.940ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
137292025-08-26T21:59:02.940ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
137302025-08-26T21:59:02.940ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
137312025-08-26T21:59:02.940ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
137322025-08-26T21:59:02.941ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 72.998325ms iteration = 27 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
137332025-08-26T21:59:02.942ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
137342025-08-26T21:59:02.942ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
137352025-08-26T21:59:02.942ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
137362025-08-26T21:59:02.942ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:550 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
137372025-08-26T21:59:02.942ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 74.618013ms iteration = 27 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
137382025-08-26T21:59:02.943ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
137392025-08-26T21:59:02.943ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
137402025-08-26T21:59:02.943ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
137412025-08-26T21:59:02.946ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
137422025-08-26T21:59:02.946ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
137432025-08-26T21:59:02.946ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
137442025-08-26T21:59:02.946ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
137452025-08-26T21:59:02.946ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
137462025-08-26T21:59:02.968ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
137472025-08-26T21:59:02.969ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-08-26T21:45:59.866217Z, time_modified: 2025-08-26T21:45:59.866217Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 9396aa55-de6f-4d6f-96db-d28e1c76fa9c, db_silo: Silo { identity: SiloIdentity { id: 9396aa55-de6f-4d6f-96db-d28e1c76fa9c, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-08-26T21:46:05.245521Z, time_modified: 2025-08-26T21:46:05.245521Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "e8529d720e4b0de099753dcdb373abb870af04baacd7c7dd850f5254858081af" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 9396aa55-de6f-4d6f-96db-d28e1c76fa9c, db_silo: Silo { identity: SiloIdentity { id: 9396aa55-de6f-4d6f-96db-d28e1c76fa9c, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-08-26T21:46:05.245521Z, time_modified: 2025-08-26T21:46:05.245521Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "e8529d720e4b0de099753dcdb373abb870af04baacd7c7dd850f5254858081af" }] }) }
137482025-08-26T21:59:02.969ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 136.380564ms iteration = 14 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"9396aa55-de6f-4d6f-96db-d28e1c76fa9c","tls_certs":["e8529d720e4b0de099753dcdb373abb870af04baacd7c7dd850f5254858081af"]}},"default_endpoint":{"silo_id":"9396aa55-de6f-4d6f-96db-d28e1c76fa9c","tls_certs":["e8529d720e4b0de099753dcdb373abb870af04baacd7c7dd850f5254858081af"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
137492025-08-26T21:59:02.974ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1213 zone_address = fd00:1122:3344:101::2
137502025-08-26T21:59:02.974ZDEBGnexus (MgsClient): client request background_task = nat_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
137512025-08-26T21:59:02.976ZDEBGnexus (MgsClient): client response background_task = nat_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "790aae64-311b-41d0-bdd0-4dad559f8005", "content-length": "26", "date": "Tue, 26 Aug 2025 21:59:02 GMT"} })
137522025-08-26T21:59:02.976ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1216 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
137532025-08-26T21:59:02.977ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
137542025-08-26T21:59:02.977ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
137552025-08-26T21:59:02.977ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 108.945186ms iteration = 27 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
137562025-08-26T21:59:02.977ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
137572025-08-26T21:59:02.978ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
137582025-08-26T21:59:02.988ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
137592025-08-26T21:59:02.988ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
137602025-08-26T21:59:02.993ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
137612025-08-26T21:59:02.993ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
137622025-08-26T21:59:02.995ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
137632025-08-26T21:59:02.998ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1213 zone_address = fd00:1122:3344:101::2
137642025-08-26T21:59:02.998ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
137652025-08-26T21:59:02.998ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: dd0c655a-95c2-48fc-b4c3-9733ed4b0faa, lookup_type: ById(dd0c655a-95c2-48fc-b4c3-9733ed4b0faa) } result = Ok(())
137662025-08-26T21:59:02.998ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
137672025-08-26T21:59:02.998ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
137682025-08-26T21:59:02.998ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "87e7f03b-516c-4cf8-9704-01fbb0b2009b", "content-length": "26", "date": "Tue, 26 Aug 2025 21:59:02 GMT"} })
137692025-08-26T21:59:02.998ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1216 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
137702025-08-26T21:59:03.000ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 132.096758ms iteration = 27 status = 0
137712025-08-26T21:59:03.000ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
137722025-08-26T21:59:03.002ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
137732025-08-26T21:59:03.003ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
137742025-08-26T21:59:03.004ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
137752025-08-26T21:59:03.006ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
137762025-08-26T21:59:03.006ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
137772025-08-26T21:59:03.006ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
137782025-08-26T21:59:03.006ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
137792025-08-26T21:59:03.009ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
137802025-08-26T21:59:03.009ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
137812025-08-26T21:59:03.009ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
137822025-08-26T21:59:03.011ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:407
137832025-08-26T21:59:03.011ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 143.166658ms iteration = 27 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
137842025-08-26T21:59:03.020ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
137852025-08-26T21:59:03.021ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
137862025-08-26T21:59:03.052ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
137872025-08-26T21:59:03.052ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = dfb404ab-acb8-43de-afe8-b429af0c161d dns_zone_name = oxide.test found_so_far = 0 version = 2
137882025-08-26T21:59:03.052ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
137892025-08-26T21:59:03.052ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
137902025-08-26T21:59:03.054ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
137912025-08-26T21:59:03.056ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1213 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 zone_address = fd00:1122:3344:101::2
137922025-08-26T21:59:03.056ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
137932025-08-26T21:59:03.057ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "58890c48-af5c-47ab-929e-08e4b6cd1f52", "content-length": "26", "date": "Tue, 26 Aug 2025 21:59:02 GMT"} })
137942025-08-26T21:59:03.057ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1216 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
137952025-08-26T21:59:03.057ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
137962025-08-26T21:59:03.057ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
137972025-08-26T21:59:03.057ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
137982025-08-26T21:59:03.058ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
137992025-08-26T21:59:03.059ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = dfb404ab-acb8-43de-afe8-b429af0c161d dns_zone_name = oxide.test found_so_far = 4 version = 2
138002025-08-26T21:59:03.059ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
138012025-08-26T21:59:03.059ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2025-08-26 21:46:05.296205 UTC dns_group = external generation = 2
138022025-08-26T21:59:03.059ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 227.068229ms iteration = 14 status = {"generation":2}
138032025-08-26T21:59:03.081ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 212.649444ms iteration = 29 status = {"physical_disks_added":0}
138042025-08-26T21:59:03.155ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
138052025-08-26T21:59:03.156ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
138062025-08-26T21:59:03.158ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 325.941617ms iteration = 27 status = null
138072025-08-26T21:59:03.168ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
138082025-08-26T21:59:03.168ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "2dd8cff9-1433-4a7f-83a0-1649a775ea3b", "date": "Tue, 26 Aug 2025 21:59:03 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
138092025-08-26T21:59:03.168ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
138102025-08-26T21:59:03.170ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "19e0b0bb-c7d4-4526-b1a8-362921e1664a", "date": "Tue, 26 Aug 2025 21:59:03 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
138112025-08-26T21:59:03.170ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
138122025-08-26T21:59:03.172ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
138132025-08-26T21:59:03.172ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 339.993835ms iteration = 16 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
138142025-08-26T21:59:03.174ZINFOnexus (BackgroundTasks): instance reincarnation completed successfully background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:113 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0
138152025-08-26T21:59:03.174ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 315.555814ms iteration = 14 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
138162025-08-26T21:59:03.199ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 367.340351ms iteration = 27 status = {}
138172025-08-26T21:59:03.224ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
138182025-08-26T21:59:03.299ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
138192025-08-26T21:59:03.300ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "a339c57c-686e-45c3-a27a-055892db2f1b", "content-length": "691", "date": "Tue, 26 Aug 2025 21:59:03 GMT"} })
138202025-08-26T21:59:03.300ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
138212025-08-26T21:59:03.300ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
138222025-08-26T21:59:03.300ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
138232025-08-26T21:59:03.302ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 434.278969ms iteration = 27 status = {}
138242025-08-26T21:59:03.333ZINFOnexus (BackgroundTasks): VPC route manager sled dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
138252025-08-26T21:59:03.333ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
138262025-08-26T21:59:03.333ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "09b30b62-a9cb-43ee-a764-204a5a26b9eb", "content-length": "372", "date": "Tue, 26 Aug 2025 21:59:03 GMT"} })
138272025-08-26T21:59:03.333ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
138282025-08-26T21:59:03.333ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
138292025-08-26T21:59:03.348ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
138302025-08-26T21:59:03.348ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
138312025-08-26T21:59:03.348ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "b5c846e3-fe4b-4f4e-8e27-936ff3631100", "date": "Tue, 26 Aug 2025 21:59:03 GMT"} })
138322025-08-26T21:59:03.348ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
138332025-08-26T21:59:03.348ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
138342025-08-26T21:59:03.351ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
138352025-08-26T21:59:03.351ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
138362025-08-26T21:59:03.354ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
138372025-08-26T21:59:03.355ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
138382025-08-26T21:59:03.358ZINFOnexus (BackgroundTasks): VPC route manager sled dd0c655a-95c2-48fc-b4c3-9733ed4b0faa push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
138392025-08-26T21:59:03.358ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
138402025-08-26T21:59:03.359ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "c46d47cc-c890-4d36-aa76-e1018d609f4d", "date": "Tue, 26 Aug 2025 21:59:03 GMT"} })
138412025-08-26T21:59:03.360ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 492.105363ms iteration = 27 status = {}
138422025-08-26T21:59:03.365ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
138432025-08-26T21:59:03.365ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "8ee498ac-689d-4071-ba88-fca4cbe696ac", "date": "Tue, 26 Aug 2025 21:59:03 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
138442025-08-26T21:59:03.365ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
138452025-08-26T21:59:03.367ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "13f2f475-865e-4b66-b122-da03643f118e", "date": "Tue, 26 Aug 2025 21:59:03 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
138462025-08-26T21:59:03.367ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
138472025-08-26T21:59:03.369ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "c1e7e603-e43f-4746-93e2-e413e2ea7d46", "date": "Tue, 26 Aug 2025 21:59:03 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
138482025-08-26T21:59:03.369ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
138492025-08-26T21:59:03.371ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
138502025-08-26T21:59:03.371ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 539.076461ms iteration = 16 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
138512025-08-26T21:59:03.404ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
138522025-08-26T21:59:03.404ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
138532025-08-26T21:59:03.407ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: d24c5c55-3a21-4a91-aeb4-45362e822df6, rack_id: e37a46f2-f789-4ca4-9841-f8c6dd705964, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(88d84f37-a4c8-4abc-af1f-27987b0d39d9) } rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0
138542025-08-26T21:59:03.407ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
138552025-08-26T21:59:03.407ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
138562025-08-26T21:59:03.417ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1832 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
138572025-08-26T21:59:03.417ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1859 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 88d84f37-a4c8-4abc-af1f-27987b0d39d9
138582025-08-26T21:59:03.418ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
138592025-08-26T21:59:03.418ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
138602025-08-26T21:59:03.418ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
138612025-08-26T21:59:03.418ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
138622025-08-26T21:59:03.418ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
138632025-08-26T21:59:03.418ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2006 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
138642025-08-26T21:59:03.418ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:408 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
138652025-08-26T21:59:03.418ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2042 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
138662025-08-26T21:59:03.418ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
138672025-08-26T21:59:03.418ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
138682025-08-26T21:59:03.418ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
138692025-08-26T21:59:03.421ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0
138702025-08-26T21:59:03.421ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
138712025-08-26T21:59:03.525ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "4094521a-40c8-445f-bcdf-07779d3e72b5", "date": "Tue, 26 Aug 2025 21:59:03 GMT"} })
138722025-08-26T21:59:03.525ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch = Switch0
138732025-08-26T21:59:03.525ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
138742025-08-26T21:59:03.525ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "86d5f8b1-09b7-4873-abbf-cbedd5d1f1ee", "content-length": "682", "date": "Tue, 26 Aug 2025 21:59:03 GMT"} })
138752025-08-26T21:59:03.526ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
138762025-08-26T21:59:03.526ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
138772025-08-26T21:59:03.530ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
138782025-08-26T21:59:03.530ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
138792025-08-26T21:59:03.533ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
138802025-08-26T21:59:03.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
138812025-08-26T21:59:03.534ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
138822025-08-26T21:59:03.535ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
138832025-08-26T21:59:03.542ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 673.751698ms iteration = 27 status = {}
138842025-08-26T21:59:05.351ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 314 local_addr = [fd00:1122:3344:101::c]:42016 method = GET remote_addr = [fd00:1122:3344:101::d]:37550 req_id = 65acd437-5c0e-4e7e-bdf2-ed5f4dfed6a5 response_code = 200 uri = /5d0959e5-47a6-4bf3-a52b-57950b9d900d
138852025-08-26T21:59:07.867ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 158 reason = Timeout
138862025-08-26T21:59:07.868ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
138872025-08-26T21:59:07.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
138882025-08-26T21:59:07.870ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
138892025-08-26T21:59:07.873ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
138902025-08-26T21:59:07.873ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
138912025-08-26T21:59:07.873ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
138922025-08-26T21:59:07.876ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
138932025-08-26T21:59:07.876ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 8.573933ms iteration = 158 status = {"chicken_switches_updated":false}
138942025-08-26T21:59:12.868ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 80 reason = Timeout
138952025-08-26T21:59:12.868ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
138962025-08-26T21:59:12.868ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 159 reason = Timeout
138972025-08-26T21:59:12.868ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
138982025-08-26T21:59:12.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
138992025-08-26T21:59:12.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
139002025-08-26T21:59:12.870ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
139012025-08-26T21:59:12.870ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 80 reason = Timeout
139022025-08-26T21:59:12.870ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
139032025-08-26T21:59:12.871ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
139042025-08-26T21:59:12.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
139052025-08-26T21:59:12.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
139062025-08-26T21:59:12.872ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
139072025-08-26T21:59:12.875ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
139082025-08-26T21:59:12.875ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
139092025-08-26T21:59:12.875ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
139102025-08-26T21:59:12.876ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
139112025-08-26T21:59:12.876ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
139122025-08-26T21:59:12.876ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
139132025-08-26T21:59:12.877ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
139142025-08-26T21:59:12.877ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 8.629064ms iteration = 159 status = {"chicken_switches_updated":false}
139152025-08-26T21:59:12.878ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
139162025-08-26T21:59:12.878ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
139172025-08-26T21:59:12.879ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
139182025-08-26T21:59:12.882ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
139192025-08-26T21:59:12.882ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
139202025-08-26T21:59:12.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
139212025-08-26T21:59:12.899ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
139222025-08-26T21:59:12.899ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 29.065105ms iteration = 80 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
139232025-08-26T21:59:12.908ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 39.430908ms iteration = 80 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}